Failed to build working FPSS compatible OpenSSL on HP-UX

I am creating openssl-1.0.2f using openssl-fips-2.0.12 (I will cover this configuration in the following lines, but at the end of the post I will list all the configurations I tried), on HP-UX11.31 (pa-risc2 ( [HPE]: pa-risc1.1 pa-risc2.0 )). Everything is fine, but when I try to use it ( in FIPS mode ) it doesn't work.

Note . Considering the fact that it is cwd

installed in the build folder (and not the installation folder that the RPATH points to ), I need to instruct the linker where to look for libraries ( SHLIB_PATH

):

[%__OPENSSL_MACHINE_PROMPT%]> OPENSSL_FIPS=1 SHLIB_PATH=./lib ./bin/openssl version -a
2063867464:error:2D06B071:FIPS routines:FIPS_check_incore_fingerprint:fingerprint does not match segment aliasing:fips.c:224:

      

        
        
        
      

    

Note . Instead of displaying any path, I replace it with a meaningful placeholder (name starting with __OPENSSL) surrounded by characters %

(equivalent to Win env vars - do not want to create confusion if real Uv env vars might be involved).

Here's the output of the command "the same" without configuring FIPS ( OPENSSL_FIPS=1

) mode :

[%__OPENSSL_MACHINE_PROMPT%]> SHLIB_PATH=./lib ./bin/openssl version -a
OpenSSL 1.0.2f-fips  28 Jan 2016
  built on: Fri Feb 26 09:53:34 2016
  platform: hpux-parisc2-gcc
  options:  bn(64,64) rc4(ptr,char) des(ptr,risc1,16,long) blowfish(idx)
  compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS  -DDSO_DL -fPIC -D_REENTRANT -march=2.0 -O3 -DB_ENDIAN -D_REENTRANT -I%__OPENSSL_BUILD_PATH%/include
  OPENSSLDIR: "%__OPENSSL_PREFIX_DIR%"

      

        
        
        
      

    

This happens on all the machines I tried to start (including the actual mechanism I built it on):

[%__OPENSSL_BUILD_MACHINE_PROMPT%]> uname -a
HP-UX hpux1131 B.11.31 U 9000/800 629887774 unlimited-user license

      

        
        
        
      

    

gcc version (using native linker (ld_pa)):

[%__OPENSSL_BUILD_MACHINE_PROMPT%]> gcc -v
  Using built-in specs.
  Target: hppa2.0w-hp-hpux11.31
  Configured with: ../gcc-4.2.4/configure --disable-shared --with-gnu-as --with-as=%__GCC_PREFIX_PATH%/bin/as --with-ld=/bin/ld --disable-nls --enable-threads=posix --prefix=%__GCC_PREFIX_PATH% --with-local-prefix=%__GCC_PREFIX_PATH%
  Thread model: posix
  gcc version 4.2.4`

      

        
        
        
      

    

• Here's the output of the openssl-fips-2.1.12 configurator:

  ./config no-asm
  Operating system: 9000/800-hp-hpux1x
    Auto Configuring fipsonly
    Auto Configuring fipsonly
    Configuring for hpux-parisc2-gcc
    Auto Configuring fipsonly
    Configuring for hpux-parisc2-gcc
        no-asm          [option]   OPENSSL_NO_ASM
        no-bf           [option]   OPENSSL_NO_BF (skip dir)
        no-camellia     [option]   OPENSSL_NO_CAMELLIA (skip dir)
        no-cast         [option]   OPENSSL_NO_CAST (skip dir)
        no-ec_nistp_64_gcc_128 [default]  OPENSSL_NO_EC_NISTP_64_GCC_128 (skip dir)
        no-gmp          [default]  OPENSSL_NO_GMP (skip dir)
        no-idea         [option]   OPENSSL_NO_IDEA (skip dir)
        no-jpake        [experimental] OPENSSL_NO_JPAKE (skip dir)
        no-krb5         [krb5-flavor not specified] OPENSSL_NO_KRB5
        no-md2          [option]   OPENSSL_NO_MD2 (skip dir)
        no-md5          [option]   OPENSSL_NO_MD5 (skip dir)
        no-mdc2         [option]   OPENSSL_NO_MDC2 (skip dir)
        no-rc2          [option]   OPENSSL_NO_RC2 (skip dir)
        no-rc4          [option]   OPENSSL_NO_RC4 (skip dir)
        no-rc5          [option]   OPENSSL_NO_RC5 (skip dir)
        no-rfc3779      [default]  OPENSSL_NO_RFC3779 (skip dir)
        no-ripemd       [option]   OPENSSL_NO_RIPEMD (skip dir)
        no-seed         [option]   OPENSSL_NO_SEED (skip dir)
        no-srp          [forced]   OPENSSL_NO_SRP (skip dir)
        no-ssl2         [forced]   OPENSSL_NO_SSL2 (skip dir)
        no-ssl3         [forced]   OPENSSL_NO_SSL3 (skip dir)
        no-store        [experimental] OPENSSL_NO_STORE (skip dir)
        no-tls1         [forced]   OPENSSL_NO_TLS1 (skip dir)
        no-tlsext       [forced]   OPENSSL_NO_TLSEXT (skip dir)
        no-zlib         [default]
        no-zlib-dynamic [default]
  
        
  
          
          
          
        
  
      

• And here's openssl-1.0.2f's:

  ./config fips shared --prefix=%__OPENSSL_PREFIX_DIR% no-rc5 no-mdc2 no-idea -fPIC no-asm --openssldir=%__OPENSSL_PREFIX_DIR%/openssl
  Operating system: 9000/800-hp-hpux1x
    Configuring for hpux-parisc2-gcc
    Configuring for hpux-parisc2-gcc
        no-asm          [option]   OPENSSL_NO_ASM
        no-ec_nistp_64_gcc_128 [default]  OPENSSL_NO_EC_NISTP_64_GCC_128 (skip dir)
        no-gmp          [default]  OPENSSL_NO_GMP (skip dir)
        no-idea         [option]   OPENSSL_NO_IDEA (skip dir)
        no-jpake        [experimental] OPENSSL_NO_JPAKE (skip dir)
        no-krb5         [krb5-flavor not specified] OPENSSL_NO_KRB5
        no-libunbound   [experimental] OPENSSL_NO_LIBUNBOUND (skip dir)
        no-md2          [default]  OPENSSL_NO_MD2 (skip dir)
        no-mdc2         [option]   OPENSSL_NO_MDC2 (skip dir)
        no-rc5          [option]   OPENSSL_NO_RC5 (skip dir)
        no-rfc3779      [default]  OPENSSL_NO_RFC3779 (skip dir)
        no-rsax         [forced]   OPENSSL_NO_RSAX (skip dir)
        no-sctp         [default]  OPENSSL_NO_SCTP (skip dir)
        no-ssl-trace    [default]  OPENSSL_NO_SSL_TRACE (skip dir)
        no-store        [experimental] OPENSSL_NO_STORE (skip dir)
        no-unit-test    [default]  OPENSSL_NO_UNIT_TEST (skip dir)
        no-zlib         [default]
        no-zlib-dynamic [default]
  
        
  
          
          
          
        
  
      

Important Note : I said the problem is with opensl-1.0.2f + openssl-fips-2.0.12 on HP-UX11.31 om PA-RISC2. What else have I tried:

• openssl-1.0.1 X (where X = [e..p]) + openssl-fips-2.0.5
• HP-UX11.31 or HP-UX11.11 on PA-RISC2
• no-asm set flag specified / unspecified

Note . While debugging, I also changed fips_premain.c (and others) and (shocking :)), the fingerprint generated by fips_premain_dso (compiled with -DFINGERPRINT_PREMAIN_DSO_LOAD

) and the one calculated at runtime are not the same! I have also dumped the memory area (native or hex) that the fingerprint is calculated for and (of course) it is different (but I cant really figure out why).

Considering the fact that it works (or should work), even if not tested on pa-risc, but only on IA64, and an extensive google search has not shown anything really relevant, I'm 99.99 pretty sure it is machine related ( machines) in my environment.

However, can anyone give me some pointers?

@ EDIT0 . I mentioned that I reproduced the same problem on IA64; it was most likely a mistake. I recently (got the machine and) built it on HP-UX11.23 IA64 and it worked great. The only problem is that cross-architecture compatibility (build / run) is only one way: PA-RISC -> IA64.