Operating System : User Mode

Hasini Samarathunga
6 min readSep 27, 2021
Work vector created by stories —freepik.com

This article is part of a series of articles explaining the development of an x86 Operating System. You can find all of the previous articles at the end.

So far we have an operating system with a kernel that has interrupt, memory segmentation, paging, and a kernel heap. But we are yet to enable support for user-space applications. This is why in this article we are finally entering User Mode.

Since we already have everything we need to enable user mode, it might seem easy to implement User Mode. But trust me, it can be a headache if we make a mistake here. So remember to read these steps carefully.

User mode: A Recap

First, let’s recap what we know of User Mode so far. In one of my previous articles, I did a quick introduction on getting started on User Mode. You can find that article here:

But if you are too lazy to read another article, here’s what we know about User Mode so far;

Our kernel, at this moment, is running with the processor in Kernel Mode, or otherwise, know as Supervisor Mode. Kernel mode is the privileged mode where the process has unrestricted access to system resources that are otherwise not accessible to the User Mode.

So a program running in the User Mode will get rejected if it wants to do certain instructions like being able to disable interrupts or halt the processor. Therefore, what we need to do is just restrict what instructions are available.

Imagine it as a large city with walls surrounding each part of the city.

The outermost part of the city is the least privileged and has the least access to the city’s resources. This is usually all the User Applications. The next privilege level goes to the Device drivers. And lastly, the Kernel occupies the inner-city.

Each of these privilege levels is called a Ring. And we need to make sure that User Mode is given the privilege level of Ring 3.

Adding Segments for User Mode

If you can remember when we first initialized the GDT we put only 3 entries to our Global Descriptor Table. If you seemed confused, you can go back to that article read all about Memory Segmentation and the GDT:

If you are all caught up with GDT entries, we can finally start our way to User Mode.

Instead of 3 entries, we are going to add 5 entries to our Global Descriptor Table. The additional two segments will be of course User Mode segments. They are very similar to the kernel segments we added earlier but the only difference is their privilege level.

So we need a code segment selector and a data segment selector for User Mode.

So the 5 selectors are the NULL selector, a code segment selector for kernel mode, a data segment selector for kernel mode, a code segment selector for user mode, and a data segment selector for user mode.

  • 0x00: Null descriptor
  • 0x08: Kernel code segment
  • 0x10: Kernel data segment
  • 0x18: User code segment
  • 0x20: User data segment

But what about the privilege level? If you can remember we put 0x9A and 0x92 as the access_byte for the Kernel segments. But since the Descriptor Privilege Level is different (3) for User segments, we use 0xFA and 0xF2 instead.

So our code will be updated like this;

Switching to user mode

Next, we need to switch to user mode. But it’s not easy as flipping a light switch. The x86 is a bit strange because there’s no direct way to switch to user mode.

The only way to execute code with a lower privilege level than the current privilege level (CPL) is to execute an exception return instruction (IRET).

So let’s write a nice function to switch to User Mode which we can call from our kmain.

This code firstly disables interrupts, as we’re working on a critical section of code. (It’s gonna take a little more work to get inter-privilege level interrupts to work properly.)

It then sets the ds, es, fs, and gs segment selectors to our User Mode data selector — 0x23.

Here we want to return from the switch_to_user_mode() function in user mode, so to do that we don’t need to change the stack pointer. The next line saves the stack pointer in EAX, for later reference. We push our stack segment selector value (0x23), then push the value that we want the stack pointer to have after the IRET. This is the value of ESP before we changed anything on the stack (stored in EAX).

The pushf instruction pushes the current value of EFLAGS — we then push the CS selector value (0x1b). And finally, we execute iret. If everything has been set upright, we should now have a kernel that can enter user mode.

Wait..so…no interrupts?

Yes, we had to disabled interrupts before flipping the mode switch. But now we have a problem. How do we re-enable interrupts?

Executing sti in user mode will cause a general protection fault, however, if we enable interrupts before we do our IRET, we may be interrupted at a bad time.

The solution is in knowing how the sti and cli instructions work. They just set the ‘IF’ flag in EFLAGS. Wikipedia tells us that the IF flag has a mask of 0x200, so what you could do, is insert these lines just after the ‘pushf’ in the asm above.

This solution means that interrupts get reenabled atomically as IRET is executing.

Writing a C program for User Mode

Since we have a kernel that can enter user mode now, we just need to write some User Mode programs for it.

One thing we can do to make it easier to develop User Mode programs is to allow the programs to be written in C. But we need to compile them to flat binaries for this to work.

First, we need to add a few assembly code lines placed at offset 0 which calls main , our c file;

This start_user_program.s file will be complied to a start_user_program.o file. And along with our main C file, we need a linker script that places these instructions first in executable.

So now we can write any program in C or assembler (or any other language that compiles to object files linkable with ld), and it is easy to load and map for the kernel.

Summary

In this article, we first went through a brief recap on User Mode. Next, we discussed how to add User segments as entries into the Global Descriptor Table. Finally, we discussed how to switch to User Mode along with writing your very own C program to run in User Mode. In short, we now have a User Mode integrated into our Operating System.

If you want to get the complete code with all the steps explained above, you can do that from my GitHub, right here,

Hope to see you in the next article as well!

Thank you!

Reference: Helin, E., & Renberg, A. (2015). The little book about OS development

Read previous articles

--

--