Class: LogStash::Inputs::Log4j

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/inputs/log4j.rb

Overview

Deprecation Notice

NOTE: This plugin is deprecated. It is recommended that you use filebeat to collect logs from log4j.

The following section is a guide for how to migrate from SocketAppender to use filebeat.

To migrate away from log4j SocketAppender to using filebeat, you will need to make 3 changes:

1) Configure your log4j.properties (in your app) to write to a local file. 2) Install and configure filebeat to collect those logs and ship them to Logstash 3) Configure Logstash to use the beats input.

.Configuring log4j for writing to local files

In your log4j.properties file, remove SocketAppender and replace it with RollingFileAppender.

For example, you can use the following log4j.properties configuration to write daily log files.

# Your app's log4j.properties (log4j 1.2 only)
log4j.rootLogger=daily
log4j.appender.daily=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.daily.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.daily.RollingPolicy.FileNamePattern=/var/log/your-app/app.%d.log
log4j.appender.daily.layout = org.apache.log4j.PatternLayout
log4j.appender.daily.layout.ConversionPattern=%d{YYYY-MM-dd HH:mm:ss,SSSZ} %p %c{1}:%L - %m%n

Configuring log4j.properties in more detail is outside the scope of this migration guide.

.Configuring filebeat

Next, www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html[install filebeat]. Based on the above log4j.properties, we can use this filebeat configuration:

# filebeat.yml
filebeat:
  prospectors:
    -
      paths:
        - /var/log/your-app/app.*.log
      input_type: log
output:
  logstash:
    hosts: ["your-logstash-host:5000"]

For more details on configuring filebeat, see www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration.html[the filebeat configuration guide].

.Configuring Logstash to receive from filebeat

Finally, configure Logstash with a beats input:

# logstash configuration
input {
  beats {
    port => 5000
  }
}

It is strongly recommended that you also enable TLS in filebeat and logstash beats input for protection and safety of your log data..

For more details on configuring the beats input, see www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html[the logstash beats input documentation].

”‘

Read events over a TCP socket from a Log4j SocketAppender. This plugin works only with log4j version 1.x.

Can either accept connections from clients or connect to a server, depending on ‘mode`. Depending on which `mode` is configured, you need a matching SocketAppender or a SocketHubAppender on the remote side.

One event is created per received log4j LoggingEvent with the following schema:

  • ‘timestamp` => the number of milliseconds elapsed from 1/1/1970 until logging event was created.

  • ‘path` => the name of the logger

  • ‘priority` => the level of this event

  • ‘logger_name` => the name of the logger

  • ‘thread` => the thread name making the logging request

  • ‘class` => the fully qualified class name of the caller making the logging request.

  • ‘file` => the source file name and line number of the caller making the logging request in a colon-separated format “fileName:lineNumber”.

  • ‘method` => the method name of the caller making the logging request.

  • ‘NDC` => the NDC string

  • ‘stack_trace` => the multi-line stack-trace

Also if the original log4j LoggingEvent contains MDC hash entries, they will be merged in the event as fields.

Defined Under Namespace

Classes: Log4jInputStream

Instance Method Summary collapse

Constructor Details

#initialize(*args) ⇒ Log4j

Returns a new instance of Log4j.



120
121
122
 
# File 'lib/logstash/inputs/log4j.rb', line 120

def initialize(*args)
  super(*args)
end

Instance Method Details

#add_socket_mixin(socket) ⇒ Object 



257
258
259
 
# File 'lib/logstash/inputs/log4j.rb', line 257

def add_socket_mixin(socket)
  socket.instance_eval { class << self; include ::LogStash::Util::SocketPeer end }
end

#build_client_socket ⇒ Object

def run



251
252
253
254
255
 
# File 'lib/logstash/inputs/log4j.rb', line 251

def build_client_socket
  s = TCPSocket.new(@host, @port)
  add_socket_mixin(s)
  s
end

#create_event(log4j_obj) ⇒ Object 



142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/logstash/inputs/log4j.rb', line 142

def create_event(log4j_obj)
  # NOTE: log4j_obj is org.apache.log4j.spi.LoggingEvent
  event = LogStash::Event.new("message" => log4j_obj.getRenderedMessage)
  event.set("timestamp", log4j_obj.getTimeStamp)
  event.set("path", log4j_obj.getLoggerName)
  event.set("priority", log4j_obj.getLevel.toString)
  event.set("logger_name", log4j_obj.getLoggerName)
  event.set("thread", log4j_obj.getThreadName)
  event.set("class", log4j_obj.getLocationInformation.getClassName)
  event.set("file", log4j_obj.getLocationInformation.getFileName + ":" + log4j_obj.getLocationInformation.getLineNumber)
  event.set("method", log4j_obj.getLocationInformation.getMethodName)
  event.set("NDC", log4j_obj.getNDC) if log4j_obj.getNDC
  event.set("stack_trace", log4j_obj.getThrowableStrRep.to_a.join("\n")) if log4j_obj.getThrowableInformation

  # Add the MDC context properties to event
  if log4j_obj.getProperties
    log4j_obj.getPropertyKeySet.each do |key|
      event.set(key, log4j_obj.getProperty(key))
    end
  end
  return event
end

#register ⇒ Object 



125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/logstash/inputs/log4j.rb', line 125

def register
  begin
    Java::OrgApacheLog4jSpi.const_get("LoggingEvent")
  rescue
    raise(LogStash::PluginLoadingError, "Log4j java library not loaded")
  end

  @logger.warn("This plugin is deprecated. Please use filebeat instead to collect logs from log4j applications.")

  if server?
    @logger.info("Starting Log4j input listener", :address => "#{@host}:#{@port}")
    @server_socket = TCPServer.new(@host, @port)
  end
  @logger.info("Log4j input")
end

#run(output_queue) ⇒ Object 



231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
 
# File 'lib/logstash/inputs/log4j.rb', line 231

def run(output_queue)
  if server?
    while !stop?
      Thread.start(@server_socket.accept) do |s|
        add_socket_mixin(s)
        @logger.debug? && @logger.debug("Accepted connection", :client => s.peer,
                      :server => "#{@host}:#{@port}")
        handle_socket(s, output_queue)
      end # Thread.start
    end # loop
  else
    while !stop?
      client_socket = build_client_socket
      @logger.debug? && @logger.debug("Opened connection", :client => "#{client_socket.peer}")
      handle_socket(client_socket, output_queue)
    end # loop
  end
rescue IOError
end

#stop ⇒ Object

method used to stop the plugin and unblock pending blocking operatings like sockets and others.



223
224
225
226
227
228
 
# File 'lib/logstash/inputs/log4j.rb', line 223

def stop
  begin
    @server_socket.close if @server_socket && !@server_socket.closed?
  rescue IOError
  end
end