Passport object empty without getting called in passport / express app without calling deserializeUser

I have an angularjs app on a different domain calling express api based requests. For example, I have an app running on localhost: 9000 and connecting to api @ localhost: 3000. Express - 3.51, Passport: 0.1.18, Angularjs: 1.2

For express, I installed cors and set the allow credentials to true:

app.use(function (req, res, next) {
  res.header('Access-Control-Allow-Credentials', true);
  res.header('Access-Control-Allow-Origin', req.headers.origin);
  res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE,PATCH');
  res.header('Access-Control-Allow-Headers', 'X-Requested-With, X-HTTP-Method-Override, Content-Type, Accept');
  next();
});

      

        
        
        
      

    

I am initializing the passport in front of the router:

app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.json());
app.use(express.urlencoded());
app.use(flash());
app.use(passport.initialize());
app.use(passport.session());*
app.use(app.router);

      

        
        
        
      

    

And in my AnguarJS frontend, I am setting defaults for using credentials in my http requests:

$httpProvider.defaults.withCredentials = true; 

      

        
        
        
      

    

Did I miss something? On login, I call serializeUser. On subsequent requests, I see that the cookie is being sent in the headers:

GET /api/shows?themes=true HTTP/1.1
Host: localhost:3000
Connection: keep-alive
Accept: application/json, text/plain, */*
Origin: http://localhost:9000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.122 Safari/537.36
Referer: http://localhost:9000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Cookie: __distillery=OHFOIWEFOWEIJFO2352039832904BLAH; connect.sid=s%lweifjeill32ij3ri3KSLIJSEF.selfijseilfjsefeljkfe%2B%2BqUeXF0k
If-None-Match: "-2134790151"

      

        
        
        
      

    

However, deserializeUser is never called because the passport object is empty on demand. Did I miss something? I also have a login page on the same domain (localhost: 3000) and the login / passport is working fine. So I think it should do something with the cross domain or header settings, but I don't know how else to debug this.

Update. If I move all my front end code to the api app (so that no cross domain occurs, authentication and session work fine ...