FluentdとFluent-Bit

Fluentd Fluent Bit
Scope Containers / Servers Embedded Linux / Containers / Servers
Language C & Ruby C
Memory ~40MB ~650KB
Performance High Performance High Performance
Dependencies Built as a Ruby Gem, it requires a certain number of gems. Zero dependencies, unless some special plugin requires them.
Plugins More than 1000 plugins available Around 70 plugins available
License Apache License v2.0 Apache License v2.0

Both Fluentd and Fluent Bit can work as Aggregators or Forwarders, they both can complement each other or use them as standalone solutions.

どちらも単独でログアグリゲーター＆フォワーダーとして機能する。

コンセプト

Event or Record

Internally, an Event always has two components (in an array form):

[TIMESTAMP, MESSAGE]

ログ行はイベントとして扱われ、イベントはタイムスタンプとメッセージのタプルで構成されている。

Filtering

In some cases it is required to perform modifications on the Events content, the process to alter, enrich or drop Events is called Filtering.

There are many use cases when Filtering is required like:

  • Append specific information to the Event like an IP address or metadata.
  • Select a specific piece of the Event content.
  • Drop Events that matches certain pattern.

フィルタリングによってイベントに大して、追加、変更、削除を行うことができる。

Tag

Most of the tags are assigned manually in the configuration. If a tag is not specified, Fluent Bit will assign the name of the Input plugin instance from where that Event was generated from.

INPUTでタグ付けを行い、FILTERやOUTPUTで利用する。

Timestamp

SECONDS.NANOSECONDS

タイムスタンプはSECONDSで示すUnixエポックタイムとNANOSECONDSで示されるナノ秒の組で表現される。

Match

Fluent Bit allows to deliver your collected and processed Events to one or multiple destinations, this is done through a routing phase. A Match represent a simple rule to select Events where it Tags matches a defined rule.

MATCHによって該当するタグを持つログを複数の宛先にルーティングするなどのルールを記述できる。

Structured Messages

{“project”: “Fluent Bit”, “created”: 1398289291}

Consider MessagePack as a binary version of JSON on steroids.

メッセージは構造化されたJSONデータ（JSONのバイナリ版であるMessagePack）を使う。