Heroku - Headless Chrome - Connection Denied
I am currently working with: Heroku Build Pack for headless chrome. https://github.com/heroku/heroku-buildpack-google-chrome/
I am facing this frenzied error where my node script (show below) cannot connect to the chrome instance. I am getting a pretty specific error:
{ Error: connect ECONNREFUSED 127.0.0.1:30555
at Object.exports._errnoException (util.js:1018:11)
at exports._exceptionWithHostPort (util.js:1041:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1090:14)
code: ‘ECONNREFUSED’,
errno: ‘ECONNREFUSED’,
syscall: ‘connect’,
address: ‘127.0.0.1’,
port: 30555 }
My node is a super simple script:
CDP((client) => {
// extract domains
// const {Network, Page} = client;
const Network = client.Network
const Page = client.Page
// setup handlers
Network.requestWillBeSent((params) => {
console.log(params.request.url);
});
Page.loadEventFired(() => {
client.close();
});
// enable events then start!
Promise.all([
Network.enable(),
Page.enable()
]).then(() => {
return Page.navigate({url: 'https://www.something.com/'});
}).catch((err) => {
console.error(err);
client.close();
});
}).on('error', (err) => {
// cannot connect to the remote endpoint
console.error(err);
});
Has anyone had any luck getting this type of job to work?
source to share
Ok I figured it out. When deploying to heroku, I used two different Procs in the Procfile. One for web
which ran the Node script. And one more to launch the ruthless chrome demon.
On Heroku, these two different procs don't even use the same dyno. This means that we are on completely different "boxes" - at least in theory. This resulted in them having different ports set in the ENV (not so that it mattered at this point - they might as well be on different continents)
Decision:
Have the Node script run the actual headless chrome and then connect to that child process using the frontend CDP
.
Also - if you're here and interested in the documentation for the interface CDP
for Node - it doesn't exist at the moment. Your best option, which is actually pretty good, is: https://chromedevtools.github.io/debugger-protocol-viewer/
Happy hunting.
Edit:
An example of how we handled starting a chrome child process from the application source
const spawn = require('child_process').spawn
spawn('/path/to/chrome/binary',[{`--remote-debugging-port=${process.env.PORT}`]) // Set by heroku
.on('close', () => console.log('CHROME_PROCESS_CLOSE'))
.on('error', e => console.log('CHROME_PROCESS_ERROR', e))
.on('exit', (e, z, a) => console.log('CHROME_PROCESS_EXIT', e, z, a))
.on('data', () => {})
source to share
Mine Procfile
looks like this to start Chrome first and then my Node.js server:
web: /app/.apt/usr/bin/google-chrome & node app/server.js
(Used in Scraping Service , a REST API for scrubbing dynamic websites. It uses headless Chrome and Cheerio.)
source to share