Fluentd user registration
I am migrating from running my containers on a Docker Swarm cluster to Kubernetes running on the Google Container Engine. When running on Docker Swarm, I configured the Docker Engine logging driver ( https://docs.docker.com/engine/admin/logging/overview/ ) to push Fluentd logs to a Fluentd container running on a Docker Swarm node with custom configuration. which then sends the Docker logs to both the Elasticsearch cluster (running Kibana) and the AWS S3 bucket. How do I migrate this to my Kubernetes nodes?
I read that I can run my Fluentd container on each node using the Daemon Set ( https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ ) but I cannot find any documentation on setting up the log driver Docker Engine for pushing Docker logs to a Fluentd container and also formatting the logs in the format I want.
source to share
We used a slightly different solution, we run fluentd as a daemonset, but docker write logs to the log and access them freely using the systemd plugin. https://github.com/reevoo/fluent-plugin-systemd . Also we are using fabric8 kubernet metadata plugin - https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter Another approach is to use tail type and / var / log / containers / * .log for path. Take a look at kubernetes_metadata_filter for some examples.
source to share