Clever Geek Handbook

Data stream protocol and pubsub emulator

I am trying to set up a development environment. Instead of using google cloud pubsub in production, I used the pubsub emulator for development and testing. To do this, I set the following environment variable:

export PUBSUB_EMULATOR_HOST=localhost:8586

This worked for google's pubsub python library, but when I switched to using the java apache beam for google data stream, the pipeline still points to google pubsub production. Is there a parameter, environment variable, or method in the pipeline that I need to set in order for the pipeline to read for the local pubsub emulator?

+3


source to share


1 answer


I found a solution in the PubsubOptions frontend and extended it to my own PipelineOptions implementation. Then using setPubsubRootUrl () set it to localhost: port of the emulator.



+4


source






More articles:


All Articles