What is the purpose of "AND AL, 0xFF"?
I am browsing a disassembled win32 C ++ program and I see quite a few:
AND AL,0xFF
Is this completely pointless or why did the compiler generate them?
Here's a longer example:
movsx eax, byte ptr [ebx]
shl eax, 18h
movsx edx, byte ptr [ebx+1]
shl edx, 10h
add eax, edx
movsx ecx, byte ptr [ebx+2]
shl ecx, 8
add eax, ecx
movsx edx, byte ptr [ebx+3]
add eax, edx
xor edx, edx
call sub_43B55C
mov ecx, eax
mov edx, eax
sar ecx, 10h
and al, 0FFh # <----
sar edx, 8
and cl, 0FFh # <----
mov [esi], cl
and dl, 0FFh # <----
mov [esi+1], dl
mov [esi+2], al
add ebx, 4
add esi, 3
inc ebp
cmp ebp, 6
jl short loc_43B5E4
The flags are not checked after these operations, so this cannot be the target. After the AND
value in AL
, CL
and DL
move to [ESI + n]
.
source to share
As @fuz suggested, this is simply a bug of the optimizer not recognizing foo & 0xff
as non-op in the context in which it was most likely used in the original function.
I have compiled the following piece of code with Borland C ++ Builder 6 after setting the compilation settings of the project to "Release":
unsigned char foobar(int foo) { return (foo >> 16) & 0xff; }
This is similar to the operations performed in the disassembly you provided close enough. We have a 32-bit value that we want to shift a certain number of bits and then turn it into a byte value, essentially returning bits 16-23 of the original value as a single byte. The input parameter has a type int
for generating the command sar
instead of shr
: most likely used in the source code as well int
.
After compiling and disassembling the resulting .obj with objconv (since I couldn't figure out how to enable the assembly of lists from within the C ++ Builder IDE), I got this:
@foobar$qi PROC NEAR
; COMDEF @foobar$qi
push ebp ; 0000 _ 55
mov ebp, esp ; 0001 _ 8B. EC
mov eax, dword ptr [ebp+8H] ; 0003 _ 8B. 45, 08
sar eax, 16 ; 0006 _ C1. F8, 10
and al, 0FFFFFFFFH ; 0009 _ 24, FF
pop ebp ; 000B _ 5D
ret ; 000C _ C3
@foobar$qi ENDP
As you can see, the redundant and
still exists. 32-bit immediate parsing can be ignored as the instruction encoding clearly shows that the immediate in the actual code stream is 8-bit: there are no other valid parameters with an 8-bit register anyway.
Microsoft Visual Studio C ++ 6 seems to be to blame for the same, but works in the whole 32-bit register (thus generating 3 more bytes due to the 32-bit immediate), clearing the upper bits - which is useless, seeing how the return value of the function was explicitly declared as 8-bit:
?foobar@@YAEH@Z PROC NEAR ; foobar
; 1 : unsigned char foobar(int foo) { return (foo >> 16) & 0xff; }
00000 55 push ebp
00001 8b ec mov ebp, esp
00003 8b 45 08 mov eax, DWORD PTR _foo$[ebp]
00006 c1 f8 10 sar eax, 16 ; 00000010H
00009 25 ff 00 00 00 and eax, 255 ; 000000ffH
0000e 5d pop ebp
0000f c3 ret 0
?foobar@@YAEH@Z ENDP ; foobar
Meanwhile, the oldest version of gcc available at godbolt compiles this correctly into what is essentially just a shift, save for natural differences between listings due to calling conventions.
source to share