What is GDT/LDT/IDT/TSS

GDT, LDT, IDT and TSS are all data structures specified by Intel x86 architecture in memory management module.

GDT, Global Descriptor Table, is used to define the characteristics of the various memory areas used during program execution, including the base address, the size and access privileges like executability and writability. These memory areas are called segments in Intel terminology. (From Wikipedia) Segment is a term for memory management in Intel x86 architecture, which is also used collaboratively with paging mechanism.

LDT, Local Descriptor Table, acts similar to GDT, which also saves segments descriptor. The main differences between GDT and LDT is: 1) GDT have only one copy in system while LDT can have many, 2) GDT may not changed during execution which LDT often changes when task switches, 3) entry of LDT is save in GDT. Entries in GDT and LDT have the same structure.

IDT, Interrupt Descriptor Table, is a data structure used by the x86 architecture to implement an interrupt vector table. The IDT is used by the processor to determine the correct response to interrupts and exceptions. Use of the IDT is triggered by three types of events: hardware interrupts, software interrupts, and processor exceptions, which together are referred to as "interrupts". The IDT consists of 256 interrupt vectors–the first 32 (0-31 or 00-1F) of which are reserved for processor exceptions. (From Wikipedia)

TSS, Task State Segment, is a special structure on x86-based computers which holds information about a task. It is used by the operating system kernel for task management. The TSS may reside anywhere in memory. A special segment register called the task register (TR) holds a segment selector that points to a valid TSS segment descriptor which resides in the GDT (a TSS descriptor may not reside in the LDT). The TSS may contain saved values of all the x86 registers. This is used for task switching. The operating system may load the TSS with the values of the registers that the new task needs and after executing a hardware task switch (such as with an IRET instruction) the x86 CPU will load the saved values from the TSS into the appropriate registers.

Note that only IDT is necessary for a CPU running normally. But actually, GDT is also used to store globally shared memory message. LDT is actually not used in Linux (every task used a shared default LDT). TSS is also used the same as LDT -- all tasks use a shared TSS. Before Linux 2.4, TSS is used for task switching. But every entry of TSS and LDT should be saved in GDT, which limits the total task numbers in the system. Linux uses software mechanism to manage task switching.

How to build a micro hypervisor

I'm trying to find out how to build a micro hypervisor these days. As to what I know, building a micro hypervisor needs at least 5 parts as follows:

1. CPU management. This part consists physical CPU (pCPU) management and virtual CPU (vCPU) management. Every pCPU should be initialed first and each CPU of VM needs a structure to management its flags, registers and states.
2. Memory management. Memory management and isolation protection is another key feature of hypervisor. Hypervisor needs to know which part of physical memory is allocated to separated VM. Some advanced feature such as buddy system and memory ballooning may also be applied in hypervisor. 
3. Interrupt control and exception handling. This part controls interrupts and exceptions caused in VM. This part mainly handles VMEXIT and inserts interrupts when VMENTER.
4. Scheduling. This is a feature to support scheduling vCPUs on pCPUs.
5. Communication. Communication between hypervisor and VM, as well as between VMs, is also another feature supported by hypervisor. Messages transferred between different domains should be handled by each sides. A message queue is used in Xen to support asynchronous communication.

In our test cases, we just need a much more simple hypervisor. We just need CPU management, memory management and interrupt control at first. Switching between different VMs can be supported in later version.

Footprint - June 25th

Actually, after all the final exams it is the first day for me to coding today. As what I have discussed with Jan last week, it seems to be a hard startup for our project.

First, I need to build up a tiny hypervisor which can support simple functions for nested virtualization test cases. Here is a simple tiny hypervisor named NOVA (https://github.com/IntelLabs/NOVA). I'm now reading through all NOVA's code. But unfortunately NOVA is written in C++! I have almost forget how to write inheritance! So I need "C++ Primer" by hand when reading codes ;(. I planned to thoroughly understand NOVA's code and write an abstract of implementing tiny hypervisor before IRC meet of this weekend.

It is sad that I have not received my welcome package! Some of my friends received request of custom declaration but I haven't got any message or phone call. I will ask Carol for my trace number of FedEx and then call FedEx for details of my package.

I'm itch for new lens, Sigma 10-20mm/F4~5.6, using on my D7000. I'm now staggering, maybe because the payment of GSoC has not come to hand.^_^

Configuring git send-email to use Gmail SMTP

Firstly, apt-get install git-email.

Then, add the correct configuration variables with the following:

$ git config --global sendemail.smtpserver smtp.gmail.com

$ git config --global sendemail.smtpserverport 587

$ git config --global sendemail.smtpencryption tls

$ git config --global sendemail.smtpuser your_email@gmail.com

Now it’s ready to use git send-email. Password is needed each time you send email.

P.S. Before using git send-email, you should use git format-patch first.

git format-patch -M origin/master
git send-email --to xxx@xxx.com --cc xxx@xxx.com 0001-xxx.patch

git config --global user.email "yzt356@gmail.com"
git config --global user.name "Arthur Chunqi Li"

Apply for GSoC’13 - Nested Virtualization Project

Email: yzt356 at gmail dot com

Short description: I’m a student of Peking University, China, and now seeking for a master degree. My research area is distributed system and virtualization. I have focused on opensource community (especially Xen & kVM) for some time an I’d like to do something for opensource.

Additional info: http://www.linkedin.com/profile/view?id=203536720

Please contact the mentor before applying so we can arrange for an IRC interview and get to know each other. Students who do not contact the mentor cannot be accepted.

Name: Arthur Chunqi Li

IRC nick: #xelatex

Phone number: +861861225XXXX (Please contact me via email)

Programming languages (fluent): C, python, assembly x86

Version control systems: Git SVN

Past open source contributions: Focus on Xen & KVM, report some bugs

Other relevant experience: I have done intern jobs in EMC and Intel about side-channel attack on virtualization and lightweight virtualization.

Why are you applying to QEMU.org?

My research area is virtualization and I have focused on Xen & KVM for some time. Some of my research work are now under KVM so I’m familiar with Qemu and KVM.

Why did you chose this project idea?

Mostly because I have great interest in it, and I think I can handle such a project.

Summary of your understanding of the project idea:

The project I apply for is “Improve nested virtualization support for x86”. which is mostly a testing work. Nested virtualization is a mechanism to run hypervisor in a virtualized environment. Nested virtualization is good for debugging hypervisor, migrating hypervisor and offering fine granularity virtualization. For KVM, nested virtualization is implemented but have a lot of bugs and unrealized features to run a nested environment. This project is to find bugs in nested environment with kvm-units-test and some other well-defined test cases. Some easy bug fixed codes should be patched for this project. The greatest challenge for this project is undetermined final goal because you don’t know what the bugs are until you find it, which is different from some programming projects. Besides testing project needs very good and detailed knowledge on the whole virtualization system as well as some hardware knowledge. Now I’m reading the manual of Intel & AMD CPU and try to get to know the vmx & svm well.

Outline of how you will tackle the project:

I will tackle the project in different steps:

1. Build up a nested virtualization environment and run some basic test cases of kvm-units-test. (Done, besides there is a bug just when I start running nested virt, I have found the reason and reported to the maillist.)

2. Reading manual and related materials related to nested virtualization.(Almost done)

3. Reading codes of kvm-units-test and try to understand the testing manners of test cases. (Doing)

4. Starting test with the current KVM codes and test cases, reporting and try to fix bugs.

5. Write special test cases (if any) for nested virtualization environment.

6. Some other works such as implementing new features (need to discussed with mentor).

How will you do the testing and the schedule.

• May 28th ~ Jun 16th. Read through the materials including Intel Manual 3C and other docs related. 
• Jun 17th ~ Jul 6th (3 weeks). VMX instructions check. This part consists of most VMX instructions (VMPTRLD, VMPTRST, VMXON, VMXOFF and so on), which is the basic of Intel VT. 
• Jul 6th ~ Jul 29th (3 weeks). MSR checks and memory checks. In this part RDMSR and WRMSR of Intel VT related MSRs will be checked. MSR is CPU vendor diversified and I will only focused on some special vendor. Memory check has been implemented in kvm-test-units and I just need to refined it. 
• Jul 30th ~ Aug 2nd (1 week). Prepare for mid-term evaluation. 
• Aug 3rd ~ Sept 7th (5 weeks).  VM-enter & VM-exit checks, memory check. VM-enter and VM-exit is the key part of Intel VT and kvm-test-units lacks of these tests. APIC checks, including I/O APIC. In the period some other undetermined tests will carry on. And I may have a trip for about a week in this period. 
• Sept 8th ~ Sept 14th (1 week). Handling some other affairs and reserve some flexible time for emergency. 
• Sept 15th ~ Sept 23th (1 week). Prepare for final evaluation.

• I’m open for any suggestions to this project, please contact me via Gmail at any time.