The C ++ Compiler changes the alignment of my structures. How can I prevent this?
I am writing code to read bitmap files.
Here is the structure I am using to read the bitmap header. See also:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd183374(v=vs.85).aspx
struct BITMAPFILEHEADER
{
WORD bfType; // 2
DWORD bfSize; // 6
WORD bfReserved1; // 8
WORD bfReserved2; // 10
DWORD bfOffBits; // 14
}; // should add to 14 bytes
If I put the following code in my main function:
std::cout << "BITMAPFILEHEADER: " << sizeof(BITMAPFILEHEADER) << std::endl;
the program prints:
BITMAPFILEHEADER: 16
It seems like realigning the data in the structure at 4-byte boundaries, presumably for efficiency. Of course this makes me unable to read the bitmap ... Even though microsoft and others point out that this is the way to do it ...
How can I prevent the structure from re-aligning?
source to share
The solution I found that works on gcc compilers, under linux:
struct BITMAPFILEHEADER
{
WORD bfType;
DWORD bfSize;
WORD bfReserved1;
WORD bfReserved2;
DWORD bfOffBits;
} __attribute__((packed));
There is probably a better, more cross compiler / platform way of doing things, but I don't know what it is.
source to share
How can I prevent the structure from re-aligning?
In C ++, I believe that you can not prevent the compiler from rearranging the "struct BITMAPFILEHEADER".
With g ++ - 5 on my Ubuntu 15.10, the same padding happens in all -O0, -O2, -O3, -Os. I have not yet found a compiler option to achieve this.
Some of the answers in the link indicated that you have no control over how the optimizations can include and complement the addition.
See also fooobar.com/questions/19603 / ...
See also Why does gcc generate 15-20% faster code if I optimize for size instead of speed?
Two other answers suggested using pragmas. Any pragma I've tried works reliably, for that particular compiler, using the options selected for that application. Pragmas are not standardized, so (IMHO) are not part of C ++. Same with compiler extensions ... C ++ developers shouldn't use either.
Simply put, C ++ is fully capable of doing what it needs to do.
It seems like realigning the data in the structure at 4-byte boundaries, presumably for efficiency.
I agree that the addition is most likely added for some concept of "compilers" (or compiler author) "efficiency". And since the structure is clearly not more economical, then perhaps the alignment allows for "faster" code. We have all the trading space for productivity, right? This is usually a good idea unless you need "packaging" that the compiler does not provide.
Below I present a possible implementation (with code) that provides absolute control over data size and padding, and this C ++ construct (no pragma, no compiler extensions) creates an object of exactly 14 bytes.
I found one comment or answer that might be similar to the approach, but it was without code.
I call this approach "Technique 1 (of 5)".
See also note 1 after the code.
Note 0 - I have all of the following code in one file (all parts are available below but reordered.).
Problem: sizeof (struct BITMAPFILEHEADER) is reported as 16 bytes:
// sizeof() reports 16 bytes -- the compiler adds pad
struct BITMAPFILEHEADER
{ // from MS docs
WORD bfType; // 2 - The file type; must be BM (bit mapped?)
DWORD bfSize; // 4 - The size, in bytes, of the bitmap file.
WORD bfReserved1; // 2 - Reserved; must be zero.
WORD bfReserved2; // 2 - Reserved; must be zero.
DWORD bfOffBits; // 4 - The offset, in bytes, from the beginning of
}; // this (14 byte) structure to the bitmap bits
this makes it impossible to read the bitmap
I think you are saying that the binary reading from your source creates an invalid object due to an alignment problem (although you are not showing your problem code).
The following proposed implementation, based on a 14 byte array, provides absolute control over data size and padding and creates an object of exactly 14 bytes.
// sizeof reports 14 bytes
class BITMAPFILEHEADER_t
{
private:
enum Constraints : size_t
{
// bf[indx] Size
Type = 0, TSz = 2, // 1 WORD
Size = 2, SSz = 4, // 2 WORD
Reserved1 = 6, RSz = 2, // 1 WORD
Reserved2 = 8, // 1 WORD
OffBits = 10, OSz = 4, // 2 WORD
//
BfSz = 14, EndConstraints
};
char bf[BfSz]; // 14 byte array
// reinterpret_cast a) explicitly allows conversion to char* from T*
// b) generates no code (compile time directive)
// vvv---short form verbose form---vvvvvvvvvvvvvvvv
char* ric (WORD* w) { return reinterpret_cast<char*>(w); }
char* ric (DWORD* dw) { return reinterpret_cast<char*>(dw); }
char* ric (BITMAPFILEHEADER_t* hdr14) { return reinterpret_cast<char*>(hdr14); }
char* ric (BITMAPFILEHEADER* hdr16) { return reinterpret_cast<char*>(hdr16); }
void memCpy (char* to, char* from, size_t count) {
for (uint i = 0; i < count; ++i) { to[i] = from[i]; }
}
// compute 'distance' between two addrs
uint64_t delta(char* highAddr, char* lowAddr) {
uint64_t dlta = (highAddr - lowAddr);
if(false) { std::cerr << "\n diag: delta is " << dlta << std::flush; }
return(dlta);
}
public:
BITMAPFILEHEADER_t() = delete;
BITMAPFILEHEADER_t(WORD utype, DWORD usize, DWORD uoff)
{
confirmFieldPlacement(); // check layout
setFields (utype, usize, uoff); // fill fields
if(false)
std::cout << "\n ctor 68 " << __PRETTY_FUNCTION__
<< std::flush;
if(false)
std::cout << "\n " << dumpFieldHex ("dumpFieldHex 91 \n");
if(false)
std::cout << " "
<< ::dumpByteHex (ric(this), BfSz, "dumpByteHex 94 \n", 6)
<< std::endl;
} // BITMAPFILEHEADER_t(w, dw, dw)
// convert
// to 14 byte from 16 byte
BITMAPFILEHEADER_t (BITMAPFILEHEADER& bmfh)
{
confirmFieldPlacement(); // ctor must check
setFields(bmfh.bfType, bmfh.bfSize, bmfh.bfOffBits);
}
~BITMAPFILEHEADER_t() = default; // all POD
// convert
// to 16 byte from 14 byte
BITMAPFILEHEADER toBMFH()
{
BITMAPFILEHEADER bmfh;
// to from count
memCpy (ric(&bmfh.bfType), &bf[Type], TSz);
memCpy (ric(&bmfh.bfSize), &bf[Size], SSz);
memCpy (ric(&bmfh.bfReserved1), &bf[Reserved1], RSz);
memCpy (ric(&bmfh.bfReserved2), &bf[Reserved2], RSz);
memcpy (ric(&bmfh.bfOffBits), &bf[OffBits], OSz);
return bmfh;
}
void setFields (WORD utype,
DWORD usize,
DWORD uoff)
{
// to from count
memCpy (&bf[Type], ric(&utype), TSz); // bfType
memCpy (&bf[Size], ric(&usize), SSz); // bfSize
DWORD zero = 0; // reserved values must be 0
memCpy (&bf[Reserved1], ric(&zero), RSz); // bfReserved1
memCpy (&bf[Reserved2], ric(&zero), RSz); // bfReserved2
memcpy (&bf[OffBits] , ric(&uoff) , OSz); // bfOffBits
} // void BITMAPFILEHEADER_t::setFields()
// reads 1x 14 bytes into bf[]
void read_14_Bytes (std::stringstream& ss14)
{
(void) ss14.read (&bf[0], 14);
// ^^^^ binary
}
// writes 1x 14 bytes from bf[]
void write_14_Bytes (std::ostream& ss14)
{
(void) ss14.write (&bf[0], 14);
// ^^^^^ binary
}
std::string dumpFieldHex(std::string lbl)
{
std::stringstream ss;
ss << lbl << std::flush << std::hex
// from len
<< ::dumpByteHex( &bf[Type], TSz, " bfType : ") // 1 WORD
<< ::dumpByteHex( &bf[Size], SSz, " bfSize : ") // 1 DWORD
<< ::dumpByteHex( &bf[Reserved1], RSz, " bfReserved1 : ") // 1 DWORD
<< ::dumpByteHex( &bf[Reserved2], RSz, " bfReserved2 : ") // 1 DWORD
<< ::dumpByteHex( &bf[OffBits], OSz, " bfOffBits : ") // 1 DWORD
<< std::dec << std::flush;
return(ss.str());
} // std::string BITMAPFILEHEADER_t::dumpFieldHex(lbl)
std::string dumpByteHex (std::string label)
{
return
::dumpByteHex (ric(this), // char* startAddr,
BfSz, // size_t len,
label, // std::string label,
0); // int indent);
} // std::string BITMAPFILEHEADER_t::dumpByteHex(lbl)
private:
void confirmFieldPlacement()
{ // distance ( to field, from start addr )
assert( delta ( &bf[Type], ric(this) ) == Type); // 0, len 2
assert( delta ( &bf[Size], ric(this) ) == Size); // 2, len 4
assert( delta ( &bf[Reserved1], ric(this) ) == Reserved1); // 6, len 2
assert( delta ( &bf[Reserved2], ric(this) ) == Reserved2); // 8, len 2
assert( delta ( &bf[OffBits], ric(this) ) == OffBits); // 10, len 4
// expected magic numbers ---------------------^^^^-----------^^
assert( sizeof(BITMAPFILEHEADER_t) == BfSz);
// sizeof whole instance 14 bytes
} // void BITMAPFILEHEADER_t::confirmFieldPlacement()
}; // class BITMAPFILEHEADER_t
The only data in BITMAPFILEHEADER_t is bf. Exactly 14 bytes. No spacers.
For a unit test, I usually collapse "main":
int main(int , char** )
{
T528_t t528; // unit test 528
Time_t start_us = HRClk_t::now();
int retVal = t528.exec(); // run unit tests
auto duration_us =
std::chrono::duration_cast<US_t>(HRClk_t::now() - start_us);
std::cout << "\n T528_t::exec() duration "
<< duration_us.count() << " us" << std::endl;
return(retVal);
}
For your overview, the code unit test "t528.exec ()" provides:
a) demo using "void read_14_Byte_records (std :: stringstream & ss14)" and "void write_14_Byte_records (std :: ostream & ss14)"
b) two conversions, between struct (16 bytes) and class (14 bytes). See also T528 :: conversion_therapy ().
c) a demo that illustrates standard techniques to make the "hidden" embedded site compiler visible.
d) a demonstration to make it visible that the class has no padding.
e) C ++ implementation of dumpByteHex ().
class T528_t
{
private:
// reinterpret_cast a) explicitly allows conversion to char* from T*
// b) generates no code (compile time directive)
// vvv---short form verbose form---vvvvvvvvvvvvvvvv
char* ric(BITMAPFILEHEADER_t* hdr14) { return(reinterpret_cast<char*>(hdr14)); }
char* ric(BITMAPFILEHEADER* hdr16) { return(reinterpret_cast<char*>(hdr16)); }
public:
T528_t()
{
std::cout << "\n " << __PRETTY_FUNCTION__ << " ctor ";
std::cout << "\n sizeof (struct BITMAPFILEHEADER ) "
<< sizeof(BITMAPFILEHEADER); // reports 16 bytes
std::cout << "\n sizeof (class BITMAPFILEHEADER_t) "
<< sizeof(BITMAPFILEHEADER_t) // reports 14 bytes
<< std::endl;
}
~T528_t() = default;
int exec()
{
std::cout << "\n " << __PRETTY_FUNCTION__ << std::endl;
{
std::stringstream ss14; // for testing binary i/o
write_14_Byte_records (ss14); // push_back some number of
// 14 byte records into ss14
read_14_Byte_records(ss14); // read/show all 14 byte records in ss14
}
struct_pad_demo(); // using struct
class_no_pad_demo(); // using class
conversion_therapy();
return(0);
} // int exec()
private: // methods
void write_14_Byte_records (std::stringstream& ss14)
{
std::cout << "\n " << __PRETTY_FUNCTION__
<< " START =" << std::flush;
{
BITMAPFILEHEADER_t hdr14 (0x1122, 0x33445566, 0x778899AA);
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " Err: hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
std::cout << "=" << std::flush; // one per record
WORD w = 0x1010;
DWORD dw = 0x10101010;
for (int i=0; i<10; ++i)
{
w = static_cast<WORD> ( w + 0x0101);
dw = static_cast<DWORD>(dw + 0x01010101);
{
BITMAPFILEHEADER_t hdr14 ( w, dw, dw );
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " Err: hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
// one per record:
std::cout << ((1 == (i % 2)) ? '=' : '.') << std::flush;
}
{
BITMAPFILEHEADER_t hdr14 (0x2211, 0x66554433, 0xAA998877);
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
std::cout << "=" << std::flush; // one per record
std::cout << "\n " << __PRETTY_FUNCTION__
<< " END =============\n" << std::endl;
} // void T528::write_14_Byte_records (std::stringstream&)
void read_14_Byte_records (std::stringstream& ss14)
{
std::cout << " " << __PRETTY_FUNCTION__;
for (int i = 0; true; ++i) // infinite loop, break out at ss14.eof()
{
// construct a work buffer to receive stream data
BITMAPFILEHEADER_t workBuff(0, 0, 0);
if(false) { // diag only, buff contents after ctor, before read_14_Bytes()
std::cout << "\n " << workBuff.dumpFieldHex("pre-read:\n"); // err 8,9?
std::cout << " " << workBuff.dumpByteHex ("pre-read:\n ");
}
// binary read of 14 byte data into workBuff
workBuff.read_14_Bytes (ss14);
if(ss14.eof()) { // no more data!
std::cout << "\n\n ss14.eof() after " << std::dec << i
<< " records." << std::endl;
break;
}
assert(ss14.good()); // tbr
std::cout << "\n\n post-read hex-dump of record [" << std::dec
<< i << "] (from ss14)" << std::endl;
std::cout << " " << workBuff.dumpFieldHex("fields within class\n");
std::cout << " " << workBuff.dumpByteHex ("class\n ");
} // for () ss14.eof()
// stringstream has no member named ‘close’.
} // void T528::read_14_Byte_records (std::stringstream&)
void struct_pad_demo()
{
std::cout << "\n\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER hdr16; // a struct
// default ctor does nothing, so data is uninitialized
{
size_t sz = sizeof(BITMAPFILEHEADER); // 16
// show uninitialized
std::cout << "\n "
<< dumpByteHex ( ric(&hdr16), sz, " uninitialized struct hdr16"
" may have 'noise'\n ")
<< std::endl;
// because all elements are POD,
// we can use std::memset to fill hdr16 with 0xff
// to val count
std::memset(&hdr16, 0xff, sz);
std::cout << " "
<< dumpByteHex(ric(&hdr16), sz,
" struct after memset(0xff) \n ")
<< std::endl;
// zero the fields
hdr16.bfType = 0; // 2
hdr16.bfSize = 0; // 4
hdr16.bfReserved1 = 0; // 2
hdr16.bfReserved2 = 0; // 2
hdr16.bfOffBits = 0; // 4
// NO visible pad, so any pad is still 0xff
std::cout << " "
<< dumpByteHex( ric(&hdr16), sz,
" struct fields zero'd (no field named"
" 'pad', so any such stay 0xff ) \n ")
<< " ^^ ^^" << std::endl;
// fields to unique values
hdr16.bfType = 0x1122; // 2
hdr16.bfSize = 0x33445566; // 4
hdr16.bfReserved1 = 0xdead; // 2
hdr16.bfReserved2 = 0xbeef; // 2
hdr16.bfOffBits = 0x778899AA; // 4
std::cout << " "
<< dumpByteHex(ric(&hdr16), sz,
" struct fields unique, pad still 0xff"
"\n ")
<< " ^^ ^^" << std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
}
} // void T528::struct_pad_demo()
void class_no_pad_demo()
{
std::cout << "\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER_t hdr14(0, 0, 0); // 14
// dummy field values--^--^--^ as fill
{
size_t sz = sizeof(BITMAPFILEHEADER_t); // 14
// show dummy initialized
std::cout << "\n "
<< dumpByteHex(ric(&hdr14), sz, " class hdr14 after ctor,"
" fields have default values \n ")
<< std::endl;
// because all elements are POD,
// we can use std::memset to fill hdr14 with 0xff
// to val count
std::memset (&hdr14, 0xff, sz); // works on class, also
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class after memset(0xff) \n ")
<< std::endl;
// zero the private fields (no pad, and bfReserved zero'd)
hdr14.setFields(0,0,0); // bfType, bfSize, bfOffBits
// no pads, no 0xff shown
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class fields zero'd (no 0xff shows there are no pads), 14 bytes\n ")
<< std::endl;
// fields to unique values
hdr14.setFields(0x2211, 0x66554433, 0xAA998877);
// bfType bfSize bfOffBits
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class fields unique, reserved are 0\n ")
<< std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
}
} // void T528::class_no_pad_demo()
void conversion_therapy()
{
std::cout << "\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER hdr16; // a struct
std::memset(&hdr16, 0xff, 16); // fill with 0xff
BITMAPFILEHEADER_t hdr14 (hdr16); // conversion ctor
std::cout << "\n "
<< dumpByteHex(ric(&hdr14), 14,
" class fields filled from struct of 0xff, "
"reserved are 0, no pad\n ")
<< std::endl;
// unique private fields (no pad, and bfReserved zero'd)
hdr14.setFields(0x1234, 0x56789abc, 0xdef0beef); // bfType, bfSize, bfOffBits
hdr16 = hdr14.toBMFH();
std::cout << " "
<< dumpByteHex(ric(&hdr16), 16,
" struct fields filled from unique fields class, "
"reserved are 0, pad un-touched, may be garbage \n ")
<< " ^^ ^^---may be garbage, but not accessible in struct \n" << std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
} // void T528::conversion_therapy()
}; // class T528_t
Refactor Reference: Includes both the C ++ dumpByteHex () utility function as well as sequence hints for stitching this code together in order
#include <chrono>
// 'reduced' chrono footprint --------------vvvvvvv
typedef std::chrono::high_resolution_clock HRClk_t; // std-chrono-hi-res-clk
typedef HRClk_t::time_point Time_t;
typedef std::chrono::milliseconds MS_t; // std-chrono-milliseconds
typedef std::chrono::microseconds US_t; // std-chrono-microseconds
typedef std::chrono::nanoseconds NS_t; // std-chrono-nanoseconds
using namespace std::chrono_literals; // suffixes include 100ms, 2s, 30us
#include <iostream> // std::cout, std::cerr
#include <iomanip> // setw(), setfill()
#include <sstream>
#include <string>
#include <cstring> // memset
#include <cassert> // assert
// forward
std::string dumpByteHex (char* startAddr,
size_t len,
std::string label = "",
int indent = 0);
// Linux substitutes
typedef uint16_t WORD;
typedef uint32_t DWORD;
// reports 16 bytes -- compiler adds unlabled pad
struct BITMAPFILEHEADER ....
// sizeof() reports 14 bytes
class BITMAPFILEHEADER_t ....
//int main(int argc, char* argv[])
int main(int , char** ) ....
// C++ support function
std::string dumpByteHex (char* startAddr, // reinterpret_cast explicitly
size_t len, // allows to char* from T*
std::string label, // = "",
int indent) // = 0
{
std::stringstream ss;
if(len == 0) {
std::cerr << "\n dumpByteHex() err: data length is 0? " << std::endl << std::dec;
assert(len != 0);
}
// Output description
ss << label << std::flush;
unsigned char* kar = reinterpret_cast<unsigned char*>(startAddr); // signed to unsigned
// unsigned char* kar = static_cast<unsigned char*>(startAddr); // invalid cast
std::string echo; // holds input chars until eoln
size_t indx;
size_t wSpaceAdded = false;
for (indx = 0; indx < len; indx++)
{
if((indx % 16) == 0)
{
if(indx != 0) // echo is empty the first time through for loop
{
ss << " " << echo << std::endl;
echo.erase();
}
// fields are typically < 8 bytes, so skip when small
if(len > 7) {
if (indent) { ss << std::setw(indent) << " "; }
ss << std::setfill('0') << std::setw(4) << std::hex
<< indx << " " << std::flush;
} // normally show index
}
// hex code
ss << " " << std::setfill('0') << std::setw(2) << std::hex
<< static_cast<int>(kar[indx]) << std::flush;
if((indx % 16) == 7) { ss << " "; wSpaceAdded = true; } // white space for readability
// defer the echo-of-input, capture to echo
if (std::isprint(kar[indx])) { echo += kar[indx]; }
else { echo += '.'; }
}
// finish last line when < 17 characters
if (((indx % 16) != 0) && wSpaceAdded) { ss << " "; indx++; } // when white space added
while ((indx % 16) != 0) { ss << " "; indx++; } // finish line
// the last echo
ss << " " << echo << '\n';
return ss.str();
} // void dumpByteHex()
Note 1:
My code shows a typical C ++ technique that provides absolute control over the size of the data. Technique 1 above is one of five.
The ideas were used during the SCU (Shelf Controller Unit) update, which was overwhelmed by the marketing success. The SCU could no longer handle (insufficient cycle cycles / seconds) full shelves of the latest functional cards. Too many states per second, too many PM (performance monitoring) per second, too many signals per second, etc.
The default solution for these ailments is more processing power and more memory. So ... we ported the SCU application: C to C ++ (new compiler), 1 MIP (cisc) to 15 mip (risc) (new processor), 1 MB to 16 MB and new OS (vxWorks) and added Ethernet, SNMP and telnet control and access.
The main requirement is backward compatibility . The installed base of existing functional cards was large. The new SCU was supposed to work with any brand name card anywhere.
Technique 1 (of 5) is the easiest to understand and is suitable for small field counting data. It can be tiresome with large messages. The largest field number of messages is probably close to 100. This large number of fields inspired at least 2 of the other 4 methods.
Note 2:
Obviously, automation support is possible for technique 1.
Note 3:
IMHO, I think there is a rationale for using pragmas or compiler extensions ... these shortcuts can generate code to support the "bigger scheme of things". that is, it allows integration to continue, to support the development of non-communication problems, etc.
source to share