What are the risks of setting EF 4.5 EDMX "Validate on Build" to "False"?

To significantly improve build times on development machines and build server, I'm considering setting the EF 4.5 "Validate on Build" object model to "False".

What are the risks of this approach?

For this question:

• The development team has a server-side build process
• Development is active on multiple branches and changes are merged into a common branch for release
• Any developer merging to or from a branch had to open the EDMX before registering, so VS will check out the merged model