Upgrading Prism Central to 2022.9 (CMSP)

Okay, so last week i updated my Prism Central instance to 2022.9 and i thought that i would write some lessons i learned around this update since Nutanix forced us to change the PC deployment mode to use the Micro Services Pattform.

Okay. So first, i encountered a small problem that i could not enable CMSP prior to upgrading (was currently on 2022.6.0.2). According to the info that i got it was because of a LCM bug. I wast just meet by this status in Prism Central Settings -> Enable Micros Services:

All the firewall parts mentioned above was meet but i could still not enable CMSP. So i found this KB article on the Nutanix portal.

Nutanix Support & Insights

The Work around was simply to disable HTTPs in LCM settings in the Prism Central instance, and i was able to enable CMSP.

So when that was taken care off i went on with the update to 2022.9, and the update progress trough LCM was smooth.

After the update, i was experiencing some wierd issues with AD LDAP integration. We could log in using our UPN but we could not view VM consoles nor connect down to Prism Elements via Prism Central. So i poked around a bit and found this KB on the portal:

Nutanix Support & Insights

The KB only mentioned if you had UPPERCASE letters in the user-names, witch we did not have. But we had UPPERCASE letters in the UPN-Suffix in the domain.

If you take alook a nutanix user database with the command

 nuclei user.list count=9999 | grep -i user@domain.com

Then we could se that the user is stored localy with lowercase letters as user@domain.com

But if we enable a higher debug level via g-flag trough this command

allssh 'echo "--pgw_log_level=debug" > ~/config/prism.gflags'
allssh genesis stop prism
cluster start

And then we tail the prism log:

tail -f prism_gateway.log | grep -i user@

Then we could clearly see that the UPN-Suffix from AD was delivered with UPPERCASE letters.

So, the workaround was to change the UPN of all the users who needed to login to Prism Central. And after changing to a custom UPN-suffix with all lowercase letters we where able to log in and run the console of VMs, and drill down to PE from PC.

This issue is tracked internally at Nutanix under ENG-513598, and a fix is scheduled in future releases of Prism Central :)

So to conclude this. Prior to enabling CMSP, make sure that you don't have any UPPERCASE letters in either your UPN-Suffix nor your usernames. Then you'll get this kind of problems :)