Heredoc in command substitution: error with unbalanced quotes or brackets (bash and zsh)

Consider the following confusing script designed to print a single quote that works for ksh

:

#!/bin/ksh
echo "$(cat <<EOF
'
EOF
)"

      

        
        
        
      

    

Output:

'

      

        
        
        
      

    

However, when I run the same script with bash 3.2.51(1)

in OS X,

#!/bin/bash
echo "$(cat <<EOF
'
EOF
)"

      

        
        
        
      

    

bash reports the following errors:

./heredoc-within-cmdsubst: line 3: unexpected EOF while looking for matching `''
./heredoc-within-cmdsubst: line 6: syntax error: unexpected end of file

      

        
        
        
      

    

And run with zsh 5.0.2

,

#!/bin/zsh
echo "$(cat <<EOF
'
EOF
)"

      

        
        
        
      

    

zsh reports the following error:

./heredoc-within-cmdsubst:6: unmatched "

      

        
        
        
      

    

Similar errors occur when a single quote is replaced with a double quote or parenthesis. If I balance the single quote / double quote / brackets with the matching single quote / double quote / parenthesis, then the script works fine with both bash and zsh.

Is this problem just a bug (respective versions) of bash and zsh, or are there any syntax rules here?