The Problem
The other day I was attempting to fix what appeared to be a configuration issue with our farm. We were receiving version mismatch errors when running code within our farm. It turned out there was two problems. First the farm did indeed have a misconfiguration (I am writing about that one separately) and it was found one of our servers in the farm wasn’t functioning properly due to a recent patch not applying properly. This post discusses the steps I took and the end result.
Initial Steps
- I initially thought the issue was the version mismatch as we recently updated the servers to Oct 2015 CU and this farm wasn’t used much. I tried the following fixes to resolve that possible issue:
- Ran the command-line version of the SharePont Config Wizard. The wizard was run during the patch, but wanted to ensure nothing was missed.
PSConfig.exe -cmd upgrade -inplace b2b -force
- This didn’t work, so the next step was to attempt to upgrade the content database manually:
Upgrade-SPContentDatabase WSS_Content_Database
- This also didn’t work. So I finally confirmed the versions were the same running the following SQL command. Once against the config DB and once against the content DB.
SELECT [VersionId],[Version],[Id],[UserName],[TimeStamp]
,[FinalizeTimeStamp],[Mode],[ModeStack],[Updates]
,[Notes]
FROM [SharePoint_Config].[dbo].[Versions]
WHERE VersionId = ‘00000000-0000-0000-0000-000000000000’
ORDER BY Id DESC
- When I compared the latest IDs I found the DB were each at the same version
- Since the DB didn’t seem to be the culprit I moved on to security. To make life easier, I simply compared the mappings and access between the farm that wasn’t working and a farm that was. Everything was the same. So to ensure it wasn’t the issue, I gave the service account full ownership access. Error still occurred.
Bad Patch
So as I stated before, we had recently had a patch run on this farm. And by recent I mean within the last few months. This wasn’t caught before because the farm isn’t currently in active use. However, when my colleague patched the servers something when wrong with our second app server (call it SP04). For some reason, even though it showed up fine in the services list (no errors listed) it was indicated that the server was missing files while trying to patch the WFEs. I corrected this (or so I thought) by reinstalling the patch on SP04. To attempt a resolution I decided to patch all the servers again. SP03 (primary App server) patched fine. SP04 patched fine, SP01 and SP02 both failed during configuration stating again that SP04 was missing files
Error found after patching
After receiving that error message I checked the patch status of the server. It was showing files missing from SP1 and the latest patch. We had this problem once before on another farm and it took us days to fix and we had to play with registry settings. I decided this wasn’t worth the effort. I decided to remove the server from the farm, completely uninstall SharePoint and start over. This should take less time and effort. And in a normal situation it would.
Uninstalling SharePoint
So I am not going to go too heavy into details on uninstalling SharePoint. There are lots of sites out there with the instructions. Basically this is what you need to do:
- Ensure any services you only have running on the server in question are transferred over to one of the other servers (just activate them in the services on servers section in Central Admin).
- Turn off all the services still running on the problem server.
- Ensure you have the install files, license key, farm passphrase and service account passwords
- You want to make sure that you do NOT remove the farm from the server list in Central Admin. Microsoft really doesn’t want you to do it that way. They explicitly say it three times in three paragraphs.
- Uninstall SharePoint using the Control Panel of the server it is installed on.
Unable to Uninstall SharePoint 2013
I started the uninstall on the SP4 server. It took it’s time but seemed to be proceeding fine. It gets to about the finish point and completes with an error saying that setup did not complete successfully.
The error in the log was: “Error: Command: ‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\BIN\PSConfig.exe’ failed with error code: -1. Type: 8::CommandFailed.” Luckily Microsoft had a KB for this. I made the necessary changes and tried again.
It ran for a bit and then another failure. Ok fine. When in doubt, reboot. After the reboot the uninstall process seemed to continue along. This time I watched it closer since I didn’t trust it. After a couple of minutes the uninstall window displayed the error:
The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 25019.
The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 25083.
Checking the logs again this time received error 1624. This is what Microsoft has to say about that error:
ERROR_INSTALL_TRANSFORM_FAILURE – There was an error applying transforms. Verify that the specified transform paths are valid.
Well that makes sense. And what’s worse, the other error messages don’t seem to be documented anywhere. I then spent a some time researching the error I did have information on (1624) and came up with nothing. Sent a few notes out to others for some assistance and called it a night.
The next day I decided to try the uninstall again as sometimes it just needs another kick. Obviously this didn’t work either. I then spent the next few hours making minor changes to the farm around services and accounts. I am not listing them because none had any affect on the uninstall. Log files remained pretty much consistent.
Something I completely forgot to do (a bad habit of mine actually) was to check the server event logs. Sometimes there is more data there. Noticed something interesting. When the error 25019 was thrown a bit more of the message was displayed here:
“…The error code is 25019. The arguments are: SYSTEM\CurrentControlSet\Services\FIMSynchronizationService\Parameters, The system cannot find the file specified.”
Seems like the system is missing an entry in the registry the installer needs to be there. This could be the break I was looking for! I backed up the entry from one of the other servers and imported it into SP4. Import was successful. Ran the uninstaller. Things looked good. The 25019 and 25083 errors did not occur again. Installer ran for another 10 minutes or so and finished with the same message that one or more components failed. Checked the log file. Back to the 1624 error message.
By this time I had been working on this problem for over 2.5 days. It would probably now be easier to manually remove the server from the farm and re-install.
Manually Removing the Farm
This process should only be followed if you are unable to uninstall the software (as I am suffering through) or if the server is no longer available.
- From Central Admin click on Servers in Farm under System Settings.
- Beside the server, click on Remove Server
- You will receive a prompt that the recommended process is to uninstall SharePoint from the server rather than removing it here.
- Click on OK.
- The list will refresh and the server will be gone.
So what will happen next? Rename the old VM and move to a new IP address. Create a new VM and give it the same name as SP4. Assign it the same IP addresses. Install SharePoint, patch and connect to the farm.
Even though I wasn’t able to fix the actual problem, I hope some of the steps I took may help you with any similar problems you are having.
Thanks for reading!
Comments