Logging in Action: With Fluentd, Kubernetes and more by Phil Wilkins

Author:Phil Wilkins [Wilkins, Phil]
Language: eng
Format: epub, pdf
Publisher: Manning Publications Co.
Published: 2022-03-06T23:00:00+00:00

Logs linked to tag w0 (including "hello":"from worker 0") will only be linked to "worker_id":"0".

Logs linked to tag w1 (including "hello":"from worker 1-2") will only be linked to "worker_id":"1" or "worker_id":"2".

7.1.2 Worker constraints

When using workers, there are some constraints that need to be considered. This relates to how processes can share (or not) resources such as file handles. So if you allocate multiple workers to a Fluentd configuration that writes to file output, the file needs to be separated, as only one worker can use one file properly. We can solve this by setting the file path to include the worker_id; for example, path "logs/#{worker_id}/${tag}/%Y/%m/%d/logfile.out.

Sharing ports among workers can be realized when the plugin uses the server helper plugin or when the plugin can natively handle the sharing of ports. The forward plugin is an example of native port management. As each process cannot use the same port, a reliable mechanism to overcome this and select suitable ports is needed. When the server helper plugin is used, it will then allocate consecutive ports to each worker. So if we had specified the use of four workers and then defined the use of a monitor_agent plugin with a port set to 30000, then worker 0 uses port 30000, worker 1 uses 30001, worker 2 uses 30002, and so on. If you are using workers, ensure that the ports being used are well separated. Separating the ports will avoid potential port collisions because the algorithm assigns the same port to different plugin instances across multiple workers. For example, specifying ports 30000 and then 30002 to different plugins, but then introducing four workers, would see ports 30002 and 30003 trying to be used by two different plugins.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.