Stack unwind memory using new operator
I have a rather annoying error that has been worrying for a while. Here is the situation, I am creating an in-memory filesystem. I have pre-allocated blocks of data for each file that can be read and written to. To implement directories, I have a simple std :: vector object containing all the files in a directory. This vector object is at the top of the file in every directory. So in order to read and write from a directory, I read the first 16 bytes into a character buffer and typed it as a vector (16 bytes because it sizeof(vector<T>)
equals 16 on my system). In particular, the first 16 bytes are not elements of the vector, but the vectors themselves. However, the vector gets confused somehow after I exit the key function.
The following code does not throw an exception and can correctly save the vector to a character buffer that will be later received.
#include <vector>
char dblock[16];
typedef std::vector<int> Entries;
void foo() {
char buf[sizeof(Entries)];
Entries* test = new (buf)Entries();
test->push_back(0);
for (int i = 0; i < sizeof(std::vector<int>); ++i) {
dblock[i] = buf[i];
}
}
void bar() {
char buf[sizeof(Entries)];
for (int i = 0; i < sizeof(std::vector<int>); ++i) {
buf[i] = dblock[i];
}
Entries* test = (Entries*)buf;
test->back();
}
int main()
{
foo();
bar();
return 0;
}
However, as soon as the foo function is changed to include such an argument, an exception is thrown anytime I try to use iterators.
void foo(int this_arg_breaks_everything) {
char buf[sizeof(Entries)];
Entries* test = new (buf)Entries();
test->push_back(0);
for (int i = 0; i < sizeof(std::vector<int>); ++i) {
dblock[i] = buf[i];
}
}
Looking at the disassembly, I came across a problem with assmbler where the function rips its frame stack:
add esp,128h <----- After stack is reduced, vector goes to an unusable state.
cmp ebp,esp
call __RTC_CheckEsp (0D912BCh)
mov esp,ebp
pop ebp
ret
I found this code using a debugger to check if ((Entries *) dblock) -> back () returns an exception or not. Specifically, this exception is "Location where access violation was detected 0XCCCCCCCC". The location in question is std :: vector std :: _ Container_base12 :: _ Myproxy -> _ Mycont -> _ Myproxy == 0XCCCCCCCC; you can find it on line 165 of uselessness.
And yes, the error only occurs when using the new inplace operator. Using a normal new statement then writing the test to the dblock does not throw an exception.
So I concluded that the ultimate reason for this is because the compiler makes a bad stack to hide some of the memory it shouldn't be.
Edit: Changed wording for clarity. Edit2: Magic numbers explained.
source to share
In visual studio 2013 this generates errors and looking at the internal data of the vector it was pretty easy to see why. The vector highlights an inner object that makes it very heavy, this inner object in turn keeps a pointer back to the vector and therefore knows the location the vector should be. When a vector memory is moved from one location to another, the memory it occupies changes and thus the internal object now points back to memory instead, which will later be removed by the debug code.
Looking at your code, it looks like the same thing. std :: _ Container_base12 is the base class used by the vector, which has a member called myProxy. myProxy is an internal object that does the heavy lifting and returns a pointer back to the vector that contains it. When you move a vector, you cannot update that pointer, and so when you go to use the moved vector data, it tries to use myProxy, which is still trying to reference the original location of the vector that was wiped out. Since this data area has been erased, it looks in it for a pointer and instead finds "CCCCCCCC", which makes debugging code for the memory data that was destroyed. He tries to access that memory location and everything explodes.
source to share
What you are doing (serializing an opaque C ++ object with an equivalent memcpy
for a local buffer) will not work persistently, because objects vector
are deep objects with a lot of pointers to a heap of memory. However, to answer your question, why are you getting the crash.
The problem is alignment. When you try to do
char buf[sizeof(Entries)];
Entries* test = new (buf)Entries();
this assumes it buf
has the correct alignment for the object Entries
. I'm not going to claim to know the internal structure vector
, but I'm pretty sure it looks something like
class vector
{
T* start;
T* end;
other stuff
}
i.e. it's a bunch of pointers to a bunch. Pointers require register alignment, which is 8 bytes on a 64-bit machine. N byte alignment means you can split the address evenly by N. However, you allocate buf
on a stack that has no guaranteed alignment, but probably accidentally has an 8 byte alignment because that's the only thing on your local frame stack ... However, if you are declaring an argument foo
and the argument has a value int
that is 4 bytes, then it is buf
no longer 8 bytes aligned since you just added 4 bytes. Then when you try to access pointers that are not aligned, you get your crash.
As an experiment, try changing foo to
void foo(int unused1, int unused2) {
It might accidentally rebuild buf
and it might not end. However, stop what you are doing and do not do it this way.
See http://en.wikipedia.org/wiki/Data_structure_alignment for more information and this for the serialization guide: http://www.parashift.com/c++-faq/serialization.html . You might consider the Boost vector class, which can be serializable.
source to share
The simplest answer for this is UNDEFINED BEHAVIOR.
std::vector
not trivially copyable , you can't get memcpy
it from one place to another.
Another problem with your code is that it dblock
can have different alignment than std::vector
. It may crash on some processors.
The third problem is that the compiler may someday return garbage when copying buff
to dblock
. This is because you are breaking the strict alias rule .
source to share