Fluentd ubuntu에 nginx 정보 elasticsearch 연동하기

plugin 설치

fluent-gem install fluent-plugin-elasticsearch

fluentd.conf 설정

<source>
  @type tail
  # Tag all events from the Nginx access log with 'nginx.access'
  tag nginx.access
  # Path to your Nginx access log
  path /var/log/nginx/access.log
  # File to record the current reading position so Fluentd knows where to resume
  pos_file /var/log/td-agent/nginx-access.log.pos

  <parse>
    @type regexp
    # This regular expression should match your Nginx log format.
    # The example below is for a common log format.
    expression /^(?<remote_addr>\S+) - (?<remote_user>\S+) \[(?<time_local>[^\]]+)\] "(?<request>[^"]*)" (?<status>\d{3}) (?<body_bytes_sent>\d+)( "(?<http_referer>[^"]*)" "(?<http_user_agent>[^"]*)")?/
    # Specify the field in the log entry that contains the timestamp
    time_key time_local
    # The time format in the log (adjust if your log uses a different format)
    time_format %d/%b/%Y:%H:%M:%S %z
  </parse>
</source>

<match nginx.access>
  @type elasticsearch
  # The hostname or IP address of your Elasticsearch node
  host localhost
  # The port on which Elasticsearch is listening
  port 9200
  # Use logstash format so that indices are created as 'nginx-YYYY.MM.DD'
  # 접속정보
  
  logstash_format true
  # Prefix for the indices in Elasticsearch
  logstash_prefix nginx
  # Flush events every 5 seconds (adjust as needed)
  flush_interval 5s
</match>

실행

sudo fluentd -c ./fluentd.conf

프로세스 죽이기

ps aux | grep fluentd
sudo kill -9 {pid}