RSS

Custom Rails Logger to Use Azure Application Insights

Rails Logger can be custom to have multiple loggers through extend(). Azure application insights can collect app’s log through trace. In this article creates a custom Rails logger and send a copy of log to Azure application insights.

Rails Logger interface

Logger

The Ruby Logger class provides a simple but sophisticated logging utility that you can use to output messages.

Log level

Logger level define as Severity.

class Logger
  # Logging severity.
  #
  # DEBUG < INFO < WARN < ERROR < FATAL < UNKNOWN
  #
  module Severity
    # Low-level information, mostly for developers.
    DEBUG = 0
    # Generic (useful) information about system operation.
    INFO = 1
    # A warning.
    WARN = 2
    # A handleable error condition.
    ERROR = 3
    # An unhandleable error that results in a program crash.
    FATAL = 4
    # An unknown message that should always be logged.
    UNKNOWN = 5
  end
end

You can then give the Logger a level, and only messages at that level or higher will be printed.

For instance, in a production system, you may have your Logger set to INFO or even WARN. When you are developing the system, however, you probably want to know about the program’s internal state, and would set the Logger to DEBUG.

Logger::LogDevice

The best way to add a custom Logger is not extend Logger, instead extend Logger::LogDevice.

Logger::LogDevice is the device used for logging messages. For example, You can have

  • file as LogDevice
  • stdout as LogDevice
  • syslog as LogDevice
  • http API as LogDevice (e.g. Azure application insights)
  • etc.

Logger::LogDevice interface:

class Logger
  class LogDevice
   close()
   reopen(log = nil)
   write(message)
  end
end

The most important function is write(message) which actually used to log message.

Azure Application Insight for Ruby

Introduction

Azure Application Insights SDK for Ruby implements Application Insights API surface to support Ruby.

Application Insights is a service that allows developers to keep their application available, performing and succeeding. This Ruby gem will allow you to send telemetry of various kinds (event, trace, exception, etc.) to the Application Insights service where they can be visualized in the Azure Portal.

Requirements

Ruby 1.9.3 and above are currently supported by this gem.

Installation

$ gem install application_insights

Or add following line to Gemfile:

gem 'application_insights', '~> 0.5.6'

Then run:

$ bundle install

Usage

Collect logs through track_trace

require 'application_insights'
tc = ApplicationInsights::TelemetryClient.new '<YOUR INSTRUMENTATION KEY GOES HERE>'
tc.track_trace 'My trace statement', ApplicationInsights::Channel::Contracts::SeverityLevel::INFORMATION, :properties => { 'custom property' => 'some value' }
tc.flush

Collecting unhandled exceptions

require 'application_insights'

# setup unhandled exception handler
ApplicationInsights::UnhandledException.collect('<YOUR INSTRUMENTATION KEY GOES HERE>')

# raise an exception and this would be send to Application Insights Service
raise Exception, 'Oops!'

Collecting requests for rack applications

In config.ru:

# set up the TrackRequest middleware in the rackup (config.ru) file
require_relative 'config/environment'
require 'application_insights'

key = ENV['APPINSIGHTS_INSTRUMENTATIONKEY']
if key != nil
  use ApplicationInsights::Rack::TrackRequest, key
end

run Rails.application

In config/environments/production.rb and config/environments/development.rb:

# For rails, suggest to set up this middleware in application.rb
# so that unhandled exceptions from controllers are also collected
# buffer size: 8096
key = ENV['APPINSIGHTS_INSTRUMENTATIONKEY']
if key != nil
  bufSize = 8096
  config.middleware.use ApplicationInsights::Rack::TrackRequest, key, bufSize
end

Configuring asynchronous channel properties

require 'application_insights'
sender = ApplicationInsights::Channel::AsynchronousSender.new
queue = ApplicationInsights::Channel::AsynchronousQueue.new sender
channel = ApplicationInsights::Channel::TelemetryChannel.new nil, queue
tc = ApplicationInsights::TelemetryClient.new '<YOUR INSTRUMENTATION KEY GOES HERE>', channel

# flush telemetry if we have 10 or more telemetry items in our queue
tc.channel.queue.max_queue_length = 10

# send telemetry to the service in batches of 5
tc.channel.sender.send_buffer_size = 5

# the background worker thread will be active for 5 seconds before it shuts down. if
# during this time items are picked up from the queue, the timer is reset.
tc.channel.sender.send_time = 5

# the background worker thread will poll the queue every 0.5 seconds for new items
tc.channel.sender.send_interval = 0.5

Create custom Logger to send logs to Azure application insights

Extend Logger::LogDevice

Create a helper module to extend Logger::LogDevice and have a method newLogger to create a Logger backend by Azure application insights.

The best practice is not hardcode application instrument key in code. Here we read from APPINSIGHTS_INSTRUMENTATIONKEY environment variable. If APPINSIGHTS_INSTRUMENTATIONKEY is not found, do not send log to application insights.

#
# File: application_insights_helper.rb
#
require 'application_insights'

module ApplicationInsightsHelper

  LogMessage = Struct.new(:datetime, :severity, :message)

  class ApplicationInsightsLogDevice < Logger::LogDevice
    @tc = nil

    def initialize()
      print 'AppInsightsLogger.initialize'
      appInsightKey = ENV['APPINSIGHTS_INSTRUMENTATIONKEY']
      if appInsightKey != nil
        sender = ApplicationInsights::Channel::AsynchronousSender.new
        queue = ApplicationInsights::Channel::AsynchronousQueue.new sender
        channel = ApplicationInsights::Channel::TelemetryChannel.new nil, queue

        @tc = ApplicationInsights::TelemetryClient.new appInsightKey, channel

        # flush telemetry if we have 10 or more telemetry items in our queue
        @tc.channel.queue.max_queue_length = 10

        # send telemetry to the service in batches of 5
        @tc.channel.sender.send_buffer_size = 5

        # the background worker thread will be active for 5 seconds before it shuts down. if
        # during this time items are picked up from the queue, the timer is reset.
        @tc.channel.sender.send_time = 5

        # the background worker thread will poll the queue every 0.5 seconds for new items
        @tc.channel.sender.send_interval = 0.5
      end
    end

    def write(message)
      return if @tc == nil

      level = nil

      if message.severity == 'FATAL'
        level = ApplicationInsights::Channel::Contracts::SeverityLevel::CRITICAL
      elsif message.severity == 'ERROR'
        level = ApplicationInsights::Channel::Contracts::SeverityLevel::ERROR
      elsif message.severity == 'WARN'
        level = ApplicationInsights::Channel::Contracts::SeverityLevel::WARNING
      elsif message.severity == 'INFO'
        level = ApplicationInsights::Channel::Contracts::SeverityLevel::INFORMATION
      end

      # only send info and above log to app insights to reduce log size.
      return if level == nil

      @tc.track_trace "#{Process.pid} #{message.message}", level
      @tc.flush
    end

    def reopen()
      # Nothing to do here.
      self
    end

    def close()
      # Nothing to do here.
    end

    def formatter()
      proc do |severity, datetime, progname, msg|
        message = LogMessage.new
        message.datetime = datetime
        message.severity = severity
        message.message = msg
        message
      end
    end
  end

  def self.newApplicationInsightsLogger()
    logDevice = ApplicationInsightsLogDevice.new
    logger = Logger.new(logDevice)

    if logDevice.respond_to?(:formatter)
      logger.formatter = logDevice.formatter
    end

    logger
  end
end

Usage

In config/environments/production.rb, use Logger’s extend() method to add a new logger.

require_relative 'path/to/app_insights_helper'

Rails.application.configure do
   ...
   ...

   logger = AppInsightsHelper::newApplicationInsightsLogger
   config.logger.extend(ActiveSupport::Logger.broadcast(logger))

   ...
   ...

That’s it. Now log will send to Azure application insights.

References