Advanced Assembly Topics

• System Calls
• CPU Modes and Memory Management
• Kernel vs User space
• Von Neumann vs Harvard
• File Access
• Windows Topics

System Calls

A system call is a request to the working kernel. In Linux (namely 32 bit), a system call is executed when a call is made to the kernel with the int 0x80 instruction - which can also be similarly invoked through syscall (in a 32 bit Linux context) and sysenter (in a 64 bit Linux context).

Differences among syscall, sysenter, and int 0x80 are described here.

For information about syscalls in Linux, both man syscall and man syscalls provide information. Moreover, cat /usr/include/asm/unistd_32.h and cat /usr/include/asm/unistd_64.h will list available syscalls.

A list of Linux System Calls is available here

A typical "Hello World" program illustrates the use of a syscall in Assembly.

section .text                   
        global  _start          ;so the linker will point to it
			                         

_start:
                                ;write msg to stdout
    mov     edx,len             ;third argument: message length
    mov     ecx,msg             ;second argument: message 
    mov     ebx,1               ;first argument: file handle (stdout)
    mov     eax,4               ;system call number (sys_write)
    int     0x80                ;call kernel

                                ;exit
  	mov     ebx,0               ;first syscall argument: exit code
    mov     eax,1               ;system call number (sys_exit)
    int     0x80                ;call kernel

section .data                   
    msg db      "Groovy!",0xa ; the string to write
    len equ     $ - msg             ;length of msg

This translates to saving values to 32 bit registers (eax, ebx, ecx, edc) and invoking a system interrupt int 0x80 (also int 80h).

message length → edx
message → ecx
specify stdout → ebx
system call number (write) → eax

Then the kernel is called to execute the command as spelled out in the registers.

After the message is printed via stdout, a similar process happens to exit peacefully Linux style i.e. with exit code '0'.

To run the above code example, you will need to compile - i.e. using nasm

nasm -f elf64 -F dwarf -g hi.asm  

[This generates debugging symbols.]

Then link the resulting object file:

ld  -o hi hi.o

Run the file using:

./hi

Notice, you did not have to add execute permissions.

For more practice with system calls in Assembly, see here.

CPU Modes for IA-32

Historical differences between Von Neumann and Harvard Architecture.

Current understanding of Kernel vs User land.

• Real Mode

In real mode, basically any memory address can be accessed. This is necessary for boot loading and starting a kernel, but a very dangerous proposition for a running system. Memory access is limited to 1 MB.

Real mode is seen at power up or reset. There is no memory protection - outside of real mode the system differentiates between Kernel and User space.

More information about real mode can be found here and here.

• Protected Mode

This is the most common operating mode for x86 processors. During booting, the CPU is transitioned from real to protected mode. In protected mode, security is organized through rings that determine levels of access. Protected mode allows access of up to 4GB of memory. It is possible to reenter what is basically real mode through Virtual 8086 mode.

More information about protected mode , real mode, and virtual mode

• System Management Mode

There is also a system management mode used primarily for management tasks. This mode can also be used to circumvent system security.

File Handling

File handling in Assembly also requires making system calls, because files are handled through the kernel.

In a typical, standalone Assembly program - there are three primary sections

• .text - used for the actual code with a mention of global _start to inform the linker.
• .bss - used for declaring variables.
• .data - for initialized variables.

For further description on typical ASM segments / sections, see here.

The following example illustrates how to write to and then read in a file in Assembly. Notice, the permissions inform the compiler that the values are octal. Also, recall that new line characters must be manually specified.

section	.text
    global _start       
	
_start:                 
   ;create the file
    mov  eax, 8
    mov  ebx, file_name
    mov  ecx, 0o660 ; file permissions - notice the octal?
    int  0x80             
	
    mov [pointer_out], eax
    
   ; write to the file
    mov	edx,len          
    mov	ecx, msg         
    mov	ebx, [pointer_out]    
    mov	eax,4            ;system call number (sys_write)
    int	0x80            
	
   ; close the file
    mov eax, 6
    mov ebx, [pointer_out]
    
   ; print "File written"
    mov eax, 4
    mov ebx, 1
    mov ecx, msg_done
    mov edx, len_done
    int  0x80
    
   ;open the file for reading
    mov eax, 5
    mov ebx, file_name
    mov ecx, 0             ;for read only access
    mov edx, 0o600 ; read only - user 
    int  0x80
	
    mov  [pointer_in], eax
    
   ;read from file
    mov eax, 3
    mov ebx, [pointer_in]
    mov ecx, file_contents
    mov edx, 26
    int 0x80
    
   ; close the file
    mov eax, 6
    mov ebx, [pointer_in]
    int  0x80    
	
   ; print the file_contents
    mov eax, 4
    mov ebx, 1
    mov ecx, file_contents
    mov edx, 26
    int 0x80
       
    mov	eax,1             ;system call number (sys_exit)
    int	0x80              ;call kernel

section	.data
    file_name db "groovyfile.txt",0
    msg db "Grooovy", 0xA, 0xD, 0
    len equ  $-msg

    msg_done db "File written", 0xA, 0xD
    len_done equ $-msg_done

section .bss
    pointer_out resb 1
    pointer_in  resb 1
    file_contents resb  26

This example was modeled after an example here.

More explanation of file creation and file handling can be found here.

System Calls in Windows System calls in Windows are more difficult. In Linux, System Calls are basically static and never changing. In Windows, system calls change by release and are typically handled through (dll) files such as nt.dll. Some system calls for Windows have been reverse engineered - an example table can be found here.

See also: Nebbett, G. (2000). Windows NT/2000 native API reference. Sams Publishing.