Reverting a Failed Cloudera Manager Upgrade

1. Log in to the Cloudera Manager Server host.

   ssh my_cloudera_manager_server_host

2. Stop the Cloudera Manager Server.

   sudo systemctl stop cloudera-scm-server

3. Stop the Cloudera Manager Agent.

   sudo systemctl stop cloudera-scm-agent

   sudo systemctl stop cloudera-scm-supervisord

If your Cloudera Manager upgrade fails, you need to determine whether the upgrade process has successfully completed updating the schema of the Cloudera Manager database. If the schema update has begun, you must restore the Cloudera Manager database using a backup taken before you began the upgrade.

1. To determine whether the schema has been updated, examine the Cloudera Manager server logs and look for a message similar to the following: Updated Schema Version to 60000. (The version number may be different for your environment.)
   Run the following command to find the log entry:

   grep 'Updated Schema Version to ' /var/log/cloudera-scm-server/cloudera-scm-server.log

2. If required, restore the database.

   The procedure for restoring the database depends on the type of database used by Cloudera Manager.

3. If you are using the embedded PostgreSQL database, stop the Cloudera Manager Embedded PostgreSQL database:

   sudo systemctl stop cloudera-scm-server-db

   If you are not using the embedded PostgreSQL database and you attempt to stop it, you might see a message indicating that the service cannot be found.

   In case of using the embedded PostgreSQL, if you see a message that the shutdown failed, then the embedded database is still running, probably because services are connected to the Hive metastore. If the database shutdown fails due to connected services, issue the following command:

   sudo service cloudera-scm-server-db next_stop_fast
   sudo service cloudera-scm-server-db stop

Cloudera Manager needs access to a package repository that contains the updated software packages. You can choose to access the Cloudera public repositories directly, or you can download those repositories and set up a local repository to access them from within your network. If your cluster hosts do not have connectivity to the Internet, you must set up a local repository.

If you have enabled high availability for Cloudera Manager, perform the following steps on the hosts for both the active and passive instances of the Cloudera Manager server.

1. Log in to the Cloudera Manager Server host.

   ssh my_cloudera_manager_server_host

2. Remove any older files in the existing repository directory:

   RHEL / CentOS

   sudo rm /etc/yum.repos.d/cloudera*manager.repo*

   SLES

   sudo rm /etc/zypp/repos.d/cloudera*manager.repo*

   Ubuntu

   sudo rm /etc/apt/sources.list.d/cloudera*.list*

3. Create a repository file so that the package manager can locate and download the binaries.

   Do one of the following, depending on whether or not you are using a local package repository:

   • Use a local package repository. (Required when cluster hosts do not have access to the internet.) See Configuring a Local Package epository.

   • Use the Cloudera public repository

     RHEL / CentOS

     1. Create a file named /etc/yum.repos.d/cloudera-manager.repo with the following content:

        [cloudera-manager]
        name=Cloudera Manager 
        baseurl=https://archive.cloudera.com/p/cm7/<Cloudera Manager version>/redhat<OS major version>/yum/
        gpgkey =https://archive.cloudera.com/p/cm7/<Cloudera Manager version>/redhat<OS major version>/yum/RPM-GPG-KEY-cloudera
        username=changeme
        password=changeme
        gpgcheck=1
        enabled=1
        autorefresh=0
        type=rpm-md

        Replace _changeme with your username and password in the /etc/yum.repos.d/cloudera-manager.repo file.

     SLES

     1. Create a file named /etc/zypp/repos.d/cloudera-manager.repo with the following content:

        [cloudera-manager]
        name=Cloudera Manager 
        baseurl=https://archive.cloudera.com/p/cm7/<Cloudera Manager version>/sles<OS major version>/yum/
        gpgkey =https://archive.cloudera.com/p/cm7/<Cloudera Manager version>/sles<OS major version>/yum/RPM-GPG-KEY-cloudera
        username=changeme
        password=changeme
        gpgcheck=1
        enabled=1
        autorefresh=0
        type=rpm-md

     2. Replace changeme with your username and password in the /etc/zypp/repos.d/cloudera-manager.repo file.

     Ubuntu

     Debian is not a supported operating system for Cloudera Manager 6.x.

     1. Create a file named /etc/apt/sources.list.d/cloudera_manager.list with the following content:

        # Cloudera Manager <Cloudera Manager version>
        deb [arch=amd64] http://username:password@archive.cloudera.com/p/cm7/<Cloudera Manager version>/ubuntu1804/apt -cm<Cloudera Manager version> contrib

     2. Run the following command:

        sudo apt-get update

     3. Replace changeme with your username and password in the /etc/apt/sources.list.d/cloudera_manager.list file.

4. A Cloudera Manager upgrade can introduce new package dependencies. Your organization may have restrictions or require prior approval for the installation of new packages. You can determine which packages may be installed or upgraded:

   RHEL / CentOS

   yum deplist cloudera-manager-agent

   SLES

   zypper info --requires cloudera-manager-agent

   Ubuntu

   apt-cache depends cloudera-manager-agent

1. Downgrade the packages. Note: Only add cloudera-manager-server-db-2 if you are using the embedded PostgreSQL database.

   RHEL / CentOS

   sudo yum clean all
   sudo yum repolist

   sudo yum downgrade "cloudera-manager-*"

   SLES

   sudo zypper clean --all

   sudo zypper dup -r baseurl

   Ubuntu

   There is no action that downgrades Cloudera Manager to the version currently in the repository.

2. Verify that you have the correct packages installed.

   Ubuntu

   dpkg-query -l 'cloudera-manager-*'

   
   Desired=Unknown/Install/Remove/Purge/Hold
   | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
   |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
   ||/ Name                   Version                Description
   +++-======================-======================-============================================================
   ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
   ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
   ii  cloudera-manager-serve 5.15.0-0.cm...~sq The Cloudera Manager Server
   

   RHEL / CentOS / SLES

   rpm -qa 'cloudera-manager-*'

   
   cloudera-manager-server-5.15.0-...
   cloudera-manager-agent-5.15.0-...
   cloudera-manager-daemons-5.15.0-...
   cloudera-manager-server-db-2-5.15.0-...

The configuration files for the Cloudera Server are under /etc/cloudera-scm-server, while those of the Agent are under /etc/cloudera-scm-agent.

Since you have downgraded the Cloudera packages, the downgrade process will not overwrite the configuration files, and the values you set before will remain.

1. If you are using the embedded PostgreSQL database, start the database:

   sudo systemctl start cloudera-scm-server-db

2. Start the Cloudera Manager Server by running the following command:

   sudo systemctl start cloudera-scm-server

3. Once the server is up (and you can access the UI), start the Cloudera Manager Agents on all the clusters by running the following command:

   sudo systemctl start cloudera-scm-agent

   If the agent starts without errors, no response displays.
4. Start the Cloudera Management Service.
   1. Log in to the Cloudera Manager Admin Console.
   2. Select Clusters > Cloudera Management Service.
   3. Select Actions > Start.

vim /var/log/cloudera-scm-server/cloudera-scm-server.log