Bug: False “Setup Mode not enabled” on Dell firmware

[majidmazin]

Bug: False “Setup Mode not enabled” on Dell firmware

Summary

Mosby reports:
on a Dell OptiPlex 7060 system, even though the firmware is verifiably in Setup Mode.

---

Environment

• Platform: Dell OptiPlex 7060 MT
• Firmware: Dell Inc. 1.32.0
• CPU: i9-9900
• Secure Boot: Disabled
• Custom Key Management: Enabled
• Keys: All keys deleted (PK, KEK, db, dbx)

---

Verification

From UEFI Shell:

Variable RT+BS 'EFIGlobalVariable:SetupMode' DataSize = 0x01
00000000: 01

This indicates the system is in Setup Mode (SetupMode = 1).

⸻

Observed Behavior

Despite the above, Mosby exits early with:

```ERROR: Setup Mode not enabled (Attempt #1)

Analysis

From Ghidra, Mosby reads SetupMode into a buffer and validates multiple byte offsets:
```byte local_45[5];
status = GetVariable(L"SetupMode", ..., data_size, local_45);

if ((status < 0) ||
    (local_45[4] == 1) ||
    (local_45[3] != 1) ||
    (local_45[2] != 0)) {

    Print("ERROR: Setup Mode not enabled");
}

On this system:
	•	Firmware returns SetupMode as a 1-byte value (0x01)
	•	Higher buffer offsets are not meaningful but are still checked
	•	This causes a valid Setup Mode to be rejected

⸻

Expected Behavior

Mosby should accept a valid SetupMode = 1 result, e.g.:

```status == EFI_SUCCESS && local_45[0] == 1

or validate based on returned data size rather than fixed offsets.

⸻

Impact

This prevents Mosby from running on otherwise valid systems, blocking:
	•	Removal of Microsoft PCA 2011
	•	Installation of Windows UEFI CA 2023


⸻

Suggested Fix

Replace strict buffer validation with a simpler check:
```if (status != EFI_SUCCESS || local_45[0] != 1) {
    error;
}

or validate using returned data size rather than fixed offsets.

This should improve compatibility with OEM firmware implementations.

⸻

Notes
	•	Issue reproduced consistently on Dell OptiPlex 7060

⸻

[pbatard]

From Ghidra

Why???

Mosby is Open Source, and you are reporting the issue in the actual source repo:
There's no need for Ghidra, you can just have a look at

Mosby/src/variables.c

Lines 163 to 244 in c9fb016

	EFI_STATUS CheckSetupMode(VOID)
	{
	EFI_STATUS Status;
	UINT8 SecureBoot = 0, SetupMode = 0, AuditMode = 0, CustomMode = 0, SetupModeAttempts = 0;
	UINTN Size;
	INTN Sel;
	// Check the amount of times the user got into UEFI Setup to try to enable Setup
	// Mode, so that we can provide more helpful messages.
	Size = sizeof(SetupModeAttempts);
	gRT->GetVariable(MOSBY_SETUP_MODE_ATTEMPTS, &gEfiMosbyGuid, NULL, &Size, &SetupModeAttempts);
	// Now that we have (possibly) read it, delete the variable
	gRT->SetVariable(MOSBY_SETUP_MODE_ATTEMPTS, &gEfiMosbyGuid, 0, 0, NULL);
	// Get the Secure Boot, Setup Mode and Audit Mode status
	Size = sizeof(SecureBoot);
	Status = gRT->GetVariable(EFI_SECURE_BOOT_MODE_NAME, &gEfiGlobalVariableGuid, NULL, &Size, &SecureBoot);
	if (EFI_ERROR(Status))
	ReportErrorAndExit(L"This platform does not support Secure Boot.\n");
	// Also make sure that Audit Mode is not set (but don't care if the variable can't be read)
	// See https://uefi.org/specs/UEFI/2.9_A/_images/Secure_Boot_and_Driver_Signing-5.png
	// Technically we should also validate that DeployedMode is 0, but no specs compliant
	// firmware can have DeployedMode and SetupMode set simultaneously.
	Size = sizeof(AuditMode);
	gRT->GetVariable(L"AuditMode", &gEfiGlobalVariableGuid, NULL, &Size, &AuditMode);
	Size = sizeof(SetupMode);
	Status = gRT->GetVariable(EFI_SETUP_MODE_NAME, &gEfiGlobalVariableGuid, NULL, &Size, &SetupMode);
	if (EFI_ERROR(Status) || SecureBoot == SECURE_BOOT_MODE_ENABLE || SetupMode != SETUP_MODE || AuditMode != 0) {
	RecallPrint(L"ERROR: Setup Mode not enabled (Attempt #%d)\n", SetupModeAttempts + 1);
	// Unfortunately, while EDK2 makes DeletePlatformKey() publicly available, which
	// should allow us to toggle Setup Mode, most modern platforms prevent the call
	// from working outside of the UEFI firmware settings.
	// Therefore we have little choice but to ask the user to reboot into UEFI Setup
	// (which we can help with, through EFI_OS_INDICATIONS_BOOT_TO_FW_UI), and wade
	// through options that are likely to look super confusing to the non initiated.
	// To alleviate some of that, we use the nonvolatile SetupModeAttempts variable
	// that detects how many times the user has been trying to get their platform
	// into Setup Mode, and alter our dialogs as a result.
	if (SetupModeAttempts == 0)
	Sel = ConsoleYesNo(AttemptMessage1);
	else if (SetupModeAttempts <= 2)
	Sel = ConsoleYesNo(AttemptMessage2);
	else if (SetupModeAttempts <= 5)
	Sel = ConsoleYesNo(AttemptMessage3);
	else
	Sel = ConsoleYesNo(AttemptMessage4);
	RecallPrintRestore();
	if (Sel == 0) {
	// Try to set Secure Boot to Custom Mode in case it helps. However, this is
	// NOT expected to do anything, since we're in runtime mode at this stage...
	CustomMode = CUSTOM_SECURE_BOOT_MODE;
	Size = sizeof(CustomMode);
	gRT->SetVariable(EFI_CUSTOM_MODE_NAME, &gEfiCustomModeEnableGuid,
	EFI_VARIABLE_NON_VOLATILE | EFI_VARIABLE_BOOTSERVICE_ACCESS,
	Size, &CustomMode);
	// Try PK deletion, in case it actually happens to work...
	DisablePKProtection();
	if (DeletePlatformKey() == EFI_SUCCESS) {
	RecallPrint(L"NOTICE: PK deletion successful! Platform should now be in Setup Mode.\n");
	} else if (IsOsIndicationsSupported(EFI_OS_INDICATIONS_BOOT_TO_FW_UI) &&
	SetOsIndication(EFI_OS_INDICATIONS_BOOT_TO_FW_UI) == EFI_SUCCESS) {
	// Log one more attempt by the user to try to enable Setup mode
	SetupModeAttempts++;
	gRT->SetVariable(MOSBY_SETUP_MODE_ATTEMPTS, &gEfiMosbyGuid,
	EFI_VARIABLE_NON_VOLATILE | EFI_VARIABLE_BOOTSERVICE_ACCESS | EFI_VARIABLE_RUNTIME_ACCESS,
	sizeof(SetupModeAttempts), &SetupModeAttempts);
	}
	RecallPrint(L"Rebooting platform...\n");
	gBS->Stall(2000000);
	gRT->ResetSystem(EfiResetWarm, EFI_SUCCESS, 0, NULL);
	} else {
	RecallPrint(L"Reboot operation cancelled by the user\n");
	Status = EFI_ABORTED;
	}
	}
	exit:
	return Status;
	}

And you are misconstruing Ghidra's output. We have multiple one-byte variables that need to be checked, since, per UEFI specs there also exists an Audit Mode that sees the SetupMode byte variable to be set as well as the AuditMode byte variable to be set.

If both conditions are met, then the system is not in Setup Mode but in Audit Mode.

Mosby should accept a valid SetupMode = 1 result,
Replace strict buffer validation with a simpler check:

Not per specs:

Audit Mode is not Setup Mode (and actually won't work to update the Secure Boot databases) so we need to account for it, and therefore we need more than a simple check for the SetupMode byte variable here.

Your system is most likely in Audit Mode. You need to fix that. And please, before going to a whole wild trip with Ghidra, do yourself a favour and check whether the application you are using is Open Source, as you will save a lot of time not having to decompile a binary and get an incomplete source where you have to guess what's going on with the various variables.

[pbatard]

Oh, and we actually used to go with the simpler non specs-compliant approach you suggest, and had to fix it in 5a49a76 because it was actually preventing Mosby from working on, surprise, surprise, Dell platforms (#6)...

[majidmazin]

Following the conversation on the earlier thread, I used an older version of Mosby to see if it recreated the issue but it worked as intended and I was able to install Windows 11 from media created using Rufus (Windows UEFI CA 2023 compliant)

[pbatard]

Or you could have just changed your firmware options to get out of Audit Mode, which I am confident you were, and used the current version of Mosby. It seems to me you went quite out of your way not to have to admit that you were wrong and that your platform was indeed in Audit Mode and not in Setup Mode.

There is no flaw in current Mosby's logic with regards to detecting whether a platform is in Setup Mode or not, as our logic follows the specs exactly, so if Mosby told you you weren't in Setup Mode, then you weren't in Setup Mode, period. You should therefore not expect operations, that are designed to only apply when a platform is in Setup Mode, to apply when a platform isn't in Setup Mode, even if some platforms may happen to be more lax.

In short, Mosby is designed to cover all possible cases, and not provide false expectations with regards to what it can and cannot accomplish, and therefore it needs to be specs compliant, especially as we ran into issues when some platforms were in Audit Mode rather than Setup Mode. Therefore, there is no scenario where trying to point people to using older versions of Mosby, that also were more lax than the specs allow, is actually helpful when the correct course of actions for users (and for yourself) was to validate that you were indeed in Audit Mode, and locate the option to switch from Audit Mode to Setup Mode.

[Kyhze]

Or you could have just changed your firmware options to get out of Audit Mode

I have the same issue as him on some Dell laptops (Latitude series, I tested on a 5480, E5570 and 5450/5470), but the Audit/Deployed mode option is NOT exposed to the user in the UEFI settings. So we have no way to (easily) change that.

[Kyhze]

Also, from my understanding of the UEFI specs, enrolling the PK while we are in AuditMode transitions the platform back into DeployedMode as described here. The diagram you posted appears to indicate similar behavior.

So why is it a problem to enroll the keys from Audit Mode (which implies SetupMode == 1), since the platform will switch back to DeployedMode as soon as we enroll the PK?

What am I missing?

All I can think of is what you mentioned:

Audit Mode is not Setup Mode (and actually won't work to update the Secure Boot databases)

However, I had no trouble manually enrolling my keys with efitools. So my guess is that the platform is buggy and incorrectly reports that it is in Audit Mode.

That, or it's not properly enforcing RO access to the variables, which I believe is what the "Setup Mode" should be in when Audit Mode is enabled, and not RW as with User/Deployed Mode (although through authenticated variables).

The specs are not entirely clear to me about this part and whether or not the variables should be writeable when in Audit Mode.

Correct me if I'm wrong.

[pbatard]

What am I missing?

You are missing that users reported that trying to enroll certs in the Secure Boot databases was a problem on some platforms when in Audit Mode in #6 (comment).

We have had multiple reports where, when the platform is in Audit Mode, we got Security Violation when trying to write the Secure Boot variables. And those appear to get sorted once the user actually put their platforms in Setup Mode.

I don't want to put software out there that may or may not work depending on whether the platform is in Audit Mode or in actual Setup Mode, else I'm going to have to face, again, people complaining that all they get from Mosby are Security Violations, as both @Almighty188 and @djc5166 found in #6.

Also, have you seen the comment that pointed on how to get to actual Setup Mode (on Dell XPS, by making sure you have Custom Keys set, and deleting all the keys — I can't vouch how much of that can be applied to the other Dell systems you mention).

I'm not sure why Dell, which apparently does provide both Setup Mode and Audit Mode on XPS (though they do not make explicit how to get to Setup Mode), would not be providing the same options on subsequent systems, so I would strongly urge you to liaise with Dell to find out if there really is no option to place the platforms you mention in Setup Mode.

[Kyhze]

You are missing that users reported that trying to enroll certs in the Secure Boot databases was a problem on some platforms when in Audit Mode in #6 (comment).

Yet it works fine with efitools so I'm a bit confused and I'm suspecting buggy firmware. I'm not questioning Mosby. I'm questioning Dell's ability to respect the UEFI specs and implement proper firmwares.

I'm just trying to understand because I'm curious. I'm not urging you to make any changes to Mosby, as I believe it is working the way it should be, as it appears to be specs-compliant.

Also, have you seen the comment that pointed on how to get to actual Setup Mode (on Dell XPS, by making sure you have Custom Keys set, and deleting all the keys — I can't vouch how much of that can be applied to the other Dell systems you mention).

Yes, that is more or less how I do it with these systems.
The way I enroll my keys on those older systems is as follows:

1. Go into the firmware settings
2. Enable "Expert Key Management" under the Secure Boot menu
3. Click the "Delete All Keys" button
4. Save and reboot into a live Linux distro
5. Use efitools to enroll my keys in this specific order: DBX.esl > DB.esl > KEK.esl > PK.auth
6. At this point I like to verify with efi-readvar that all my keys were enrolled successfully, and if so I just reboot and turn on Secure Boot to confirm it's working as intended

I was also able to enroll keys through the firmware GUI after deleting the keys, following the same enrollment order. Tho some firmwares seem to prefer that the ESL are authenticated rather than using raw .esl files, even in Setup Mode (well... assuming it is actually in Setup Mode, you know...).
So for those you can just generate your .auth variants and follow the same enrollment order and you should be fine. This makes me confirm that the Dell firmwares are indeed a mess, as they behave very inconsistently depending on various factors, it would seem.

I'm not sure why Dell, which apparently does provide both Setup Mode and Audit Mode on XPS (though they do not make explicit how to get to Setup Mode), would not be providing the same options on subsequent systems, so I would strongly urge you to liaise with Dell to find out if there really is no option to place the platforms you mention in Setup Mode.

I'm not sure either, but it is annoying. From my observations it looks like the menu to toggle between Deployed and Audit Mode are only available on more "premium" lines of Dell products.

Thank you for your reply.

[pbatard]

I guess one thing I could do is add an explicit option in Mobsy to allow it to continue if we detect that we are in Audit Mode rather than Setup Mode, so that users who appear to be limited by their firmwares in terms of Setup vs Audit Mode, but still want to proceed (and understand that there is no guarantee that Mosby will work) can still give it a try.

The more I read the specs, the more I think that the default behaviour, for specs compliant firmwares, should be to allow the databases to be written in either mode, even though we have some evidence that some platforms do appear to restrict what can be accomplished in Audit Mode compared to what they allow in Setup Mode.

But before I do that, I'll see if I can get my hands on one of these Dell systems where Setup Mode doesn't appear to be selectable. It will probably be a while (weeks or months) before that happens.

[Kyhze]

That sounds like a reasonable idea.

I'll see if I can get my hands on one of these Dell systems where Setup Mode doesn't appear to be selectable. It will probably be a while (weeks or months) before that happens.

I would happily ship you one of those free of charge if you have a P.O. Box or something. I have plenty of those lying around, mainly some Dell Latitude E5450, which also exhibit the weird Setup Mode behavior, on top of the Deployed/Audit Modes not being accessible from the firmware UI.

[pbatard]

@Kyhze, I don't have a P.O. box and I reside in Ireland, so I'm not sure how we might be able to sort out shipping. But if you e-mail me to pete@akeo.ie, we can privately see if we can try to work something out.

[pbatard]

Okay, I'm going to reopen this issue, as I am still planning to investigate systems where only Audit Mode appears to be available, and I of course would prefer Mosby users not to be left stranded on such systems.