How to troubleshoot Cloudstack on Ubuntu Linux

Case



You face one or more issues while deploying or operating Apache Cloudstack. This article provides high-level guidance on how to troubleshoot Cloudstack on Ubuntu Linux. This article assumes the latest version of Cloudstack, which at time of this writing is 4.16.1.





Solution



Known issues you may come across with Cloudstack are the following:



- Data Loss on Exported Primary Storage

- Recovering a Lost Virtual Router

- Maintenance mode not working on vCenter

- Unable to deploy VMs from uploaded vSphere template

- Unable to power on virtual machine on VMware

- Load balancer rules fail after changing network offering

- Troubleshooting Internet Traffic

The official Apache Cloudstack troubleshooting guide offers solutions to all above known issues. You should follow this troubleshooting guide for a step-by-step approach. However you should keep in mind the following general troubleshooting hints and tips, when troubleshooting Cloudstack issues:



- Check if your issue is a known issue in the Cloudstack Apache JIRA project or in the official list of issues in the Github project: - https://issues.apache.org/jira/projects/CLOUDSTACK/summary

- https://github.com/apache/cloudstack/issues

- Investigate output from the Cloudstack management server log files. The CloudStack Management Server logs CS web tier (CS portal), application tier and database (MySQL) tier events in /var/log/cloudstack/management/. The CloudStack Agent Server logs its activities in /var/log/cloudstack/agent/.



The following commands can be used to filter these CS logs to only the records referring to a potential issue or error, stemming from either the CS management server or the CS agents (on the respective CS hosts).



grep -i -E 'exception|unable|fail|invalid|leak|warn|error' /var/log/cloudstack/management/management-server.log

grep -i -E 'exception|unable|fail|invalid|leak|warn|error' /var/log/cloudstack/agent/agent.log

grep -i -E 'exception|unable|fail|invalid|leak|warn|error' /var/log/cloudstack/agent/security_group.log

grep -i -E 'exception|unable|fail|invalid|leak|warn|error' /var/log/cloudstack/agent/setup.log



- Investigate a Cloudstack job ID. All jobs running as synchronous or asynchronous transactions in Cloudstsack have a unique job ID. You can find the job ID by doing an initial grep on the CS logs and then make a more granular grep based on the specific job ID(s) which are involved in the error under investigation. An example is shown below for job ID 1532.

grep "job-1532" /var/log/cloudstack/management/management-server.log

- Investigate CS database logs which can be queried from the CS portal user interface.



- Activate Cloudstack event log notifications by using the Cloudstack-provided AMQP client. There are two types of events logged in the CloudStack Event Log. Standard events log the success or failure of an event and can be used to identify jobs or processes that have failed. There are also long running job events for asynchronous and synchronous CS jobs. The long running job events log when a job is scheduled, when it starts, and when it completes. To configure CS AMQP for event log notifications to be sent to external apps and services, consult the following URL: https://docs.cloudstack.apache.org/en/latest/adminguide/events.html#amqp-configuration.

- Optionally, you can also enable the CS Apache Kafka framework. To configure CS Kafka for event log notifications to be sent to external apps and services, consult the following URL: https://docs.cloudstack.apache.org/en/latest/adminguide/events.html#kafka-configuration.

- If all else fails, try to send your Cloudstack error/issue to the official Cloudstack mailing lists and IRC channels: - https://cloudstack.apache.org/mailing-lists.html

- https://cwiki.apache.org/confluence/display/CLOUDSTACK/Mailing+List+Guidelines

- https://cwiki.apache.org/confluence/display/CLOUDSTACK/IRC+Guidelines

Reference



Official Apache Cloudstack troubleshooting guide: https://docs.cloudstack.apache.org/en/latest/adminguide/troubleshooting.html


https://stefanos.cloud/kb/how-to-troubleshoot-cloudstack-on-ubuntu-linux/

Comments

Post a Comment

Popular posts from this blog

Acronis Cyber Protect 15 virtual machine backup not working when using shared mode virtual disks vhds

Image
Case When you apply and run an Acronis Cyber Protect backup against a virtual machine which is part of a guest cluster which makes use of virtual disk shared files of type .vhds (VHD Set), you run across the following error. Activity succeeded with warnings. Activity 'Preparing for backup of virtual machine' succeeded with the following warning 'Windows error: (0xC05CFF0A). The requested operation cannot be performed on the virtual disk as it is currently used in shared mode. Or the error may state something along the lines of: "Failed to open virtual disk file '.vhds'. Unknown format of the virtual disk." The same error as above can occur in virtual machine backup of virtual machines which include a .vdhx shared disk. Solution The above error is by design when you are attempting a virtual machine backup to virtual machine which does not have an Acronis backup agent installed, as per https://kb.acronis.com/content/68573 . According to Acronis, as of July 2
Read more

How to perform hardware health checks in Windows

Image
Case You need to perform an overall hardware health check in a Windows machine, running either a Windows client or a Windows Server operating system. This article provides guidance on the steps to carry out to perform hardware health checks in Windows. Solution Carry out the steps below. - Thoroughly check all Windows event logs (Application, System, Security) for any warnings or errors which may be related to issues you are experiencing with your Windows machine. - Check the device management mmc console by running devmgmt.msc. Ensure that you choose to show all hidden devices from the View menu. If you notice any exclamation marks next to any device, you will need to re-install its driver from the respective manufacturer to ensure that there afterwards no warnings in the device management console. - Check the health of all your disks , either HDD or SSD. Follow instructions in the following KB article for steps to follow to check your disk health: https://stefanos.cloud/kb/how-to-ch
Read more

How to resolve Group Policy error codes 8007071a and 800706ba

Image
Case You are running "Group Policy Update" on an OU inside the Group Policy Management Console in Active Directory but you are receiving RPC errors from your servers or domain-joined workstations, as shown in the example below. After a Group Policy Update, you come across Group Policy error codes 8007071a and 800706ba. This means that the local Windows Firewall is not configured properly to allow Group Policy (GPO) traffic. GPO traffic an be either remote GPO update or remote GP Resultant Set of Policy (RSOP) operations. Solution Option 1 Run gpedit.msc and create a new local group policy object which implements the following policies under Local Computer Policy --> Computer Configuration --> Windows Settings --> Security Settings --> Windows Defender Firewall --> Inbound Rules : The full configuration is shown below. After the above local group policies are configured on each domain server and workstation, you can start applying group policy objects to the dom
Read more