Fun times with Citrix Workspace App updates
I recently had some issues with a Citrix Workspace App update (2311 to 2405) at a customer and I thought a walkthrough of the issue would make for a slightly decent blogpost. So let’s do this…
Before I begin…
If you are here just for the conclusion then here you go:
[TL;DR]
When using Citrix Workspace App with the App Protection feature enabled, don’t use the /cleaninstall parameter when updating. Or if you have to, uninstall the previous version first and reboot.
[TL;DR]
Why update?
On July 9th 2024 the Citrix Security Article CTX678036 went live. This bulletin mentions a local privilege escalation vulnerability in Citrix Workspace App versions before 2403.1 (Current Release) and before 2402 LTSR or 2203.1 LTSR CU6 Hotfix 2.
The CVE ID for this vulnerability is CVE-2024-6286.
To fix this you would simply need to update to the versions mentioned or newer. At a customer of mine this meant upgrading the Workspace App from 2311 (23.11.1.140) to 2405 (24.5.0.131) on a varied amount of systems: VDI desktops, SBC desktops, physical workstations and laptops spread over two separate domains.
The customer uses Microsoft SCCM for these kind of things, so in the first domain I copied the package for 2311, added the Citrix Workspace App 2405 offline installer and renamed the whole thing and repeated the process for the second domain.
The 2311 installations were done with the following parameters:
CitrixWorkspaceFullInstaller.exe /silent /noreboot /includeSSON ENABLE_SSON=Yes /AutoUpdateCheck=Disabled ENABLECEIP=False /includeAppProtection /forceinstall
Because I copied the package the 2405 installation was done in the same way.
So after jumping through all the change management hoops, which included testing the package in a new VDI image build as well as an update on my company laptop, we were ready to create an SCCM deployment to apply the update to all physical machines in both domains.
Users with connection issues
Later that day reports were coming in that in the second domain users were unable to start Citrix sessions. They could login to NetScaler Gateway, but when launching the session it would just timeout. After going back and forth with the networking department we determined it had to do with the update. However, in the first domain everything was fine.
Without boring you with all the troubleshooting steps we found out that all workstations in the second domain only had half an installation of the Citrix Workspace App version 2405.
In C:\Program Files (x86)\Citrix\ there were only the ICA Client and Logs folders and not the Citrix Workspace 2405 folder. So while during session launch the session started notification popped up, there were just files missing and the session could not actually be launched.
First priority was to make sure these workstations got a working Citrix Workspace App installation no matter what version. So after some ugly and creative PowerShell scripting we managed to loop through each workstation, identify if it had the problem and, if it did, install CWA version 2405 with just /silent /cleaninstall parameters. After some after-hours work we managed to fix each machine.
But then there is still the question: Why did this happen?
In my analysis I noticed that a small difference in the installation parameters used between these domains: The first domain used /forceinstall while the second used /cleaninstall. While I believe these are similar commands there appears to be a small difference:
From Citrix docs:
/ForceInstall
This switch is effective when cleaning up any existing configuration or entries of Citrix Workspace app in the system. Use this switch when upgrading from an unsupported version of Citrix Workspace app version and when the installation or upgrade is unsuccessful.
/CleanInstall
Command to cleanup any leftover traces such as files and registry values from a previous uninstall and then freshly install the new version of the Citrix Workspace app.
After testing in my home-lab setup on two cleanly installed Windows 10 VMs I found out the App Protection feature was the culprit. If this is installed you can’t perform an update with the parameter /cleaninstall. You would need to uninstall (and reboot) the old version first.
Why doesn’t the Citrix Workspace App just give a decent error about this? Good question, all I got from the logs (in C:\Program Files (x86)\Citrix\Logs) was error code 40017 or error code 40034 for which I (or rather Google) can’t find a good explanation what these codes mean.
The installation/update logs looked like this:
15:06:24: Information - CInstallationManager::InstallComponent(1056) - Component Install Commandline: REBOOT=ReallySuppress MSIDISABLERMRESTART=0 MSIRESTARTMANAGERCONTROL=0 ARPSYSTEMCOMPONENT=1 NEED_RECEIVER=n TROLLEYINSTALL=1 SILENT=1 silent="true" noreboot="true" includesson="true" enable_sson="Yes" autoupdatecheck="Disabled" enableceip="False" includeappprotection="true" cleaninstall="true" offlineinstaller="true" ALLUSERS=1 ALLOWCEFBINARYINSTALL=1 CUSTOMERREBRANDINGENABLED=0 SHORTCUTBRANDEDNAME="Citrix Workspace" ISWINDOWS11=0 APPPROTECTIONDORMANTINSTALL=0 APPDATAPROTECTIONDORMANTINSTALL=1
15:06:24: Information - CInstallationManager::InstallComponent(1082) - MSI Install from path : C:\Program Files (x86)\Citrix\Ctx-F01C7FF7-1A59-4499-8B53-3ABE31D09A1E\AppProtection.msi and IsSelected is 1
15:09:28: Information - CInstallationManager::InstallComponent(1204) - Component Install Status: ID = AppProtection, InstallMode = Install, InstallResult = 1603, InstallSuccess = False, InstallRebootRequired = False, InstallSuppressed = False
15:09:28: Information - CInstallationManager::InstallComponent(1222) - InstallComponent complete for AppProtection.
15:09:28: Information - CInstallationManager::LogCloseDownEvents(85) - sending analytics events post install/upgrade
15:09:28: Information - CInstallationManager::LogCloseDownEvents(147) - sending analytics events for failed install/upgrade
15:09:28: Information - CWAInstallerHelper::CWA_InstallerHelper::WriteLogs(28) - InstallFlow : Installation Return ErrorCode : 40034
15:09:28: Information - CWAInstallerHelper::CWA_InstallerHelper::WriteLogs(28) - InstallFlow : Got error during installation Error code : 40034
Wrapping up…
During writing this blogpost I noticed that Citrix documentation for the Workspace App actually has a notice about this:
To leverage the /CleanInstall command while App Protection is enabled, you must first uninstall the Citrix Workspace app, reboot the machine, and then initiate a fresh installation with the command /CleanInstall.
There was also an issue posted by Carl Stalhood on Twitter/X that mentioned something similar (link) in terms of the errors that the users received, but the cause seems something different (and will be fixed by a hotfix).
It’s a shame that I haven’t noticed this notice before. Oh well, a day in the life of someone in IT.