Class: Batsd::Truncator

Inherits:
Object
  • Object
show all
Defined in:
lib/batsd/truncator.rb

Overview

Handle truncation for redis zsets and files written to disk

Instance Method Summary (collapse)

Constructor Details

- (Truncator) initialize(options = {})

Create a new truncator

  • Establish the diskstore that will be used

  • Establish the redis connection that will be needed



12
13
14
15
16
17
 
# File 'lib/batsd/truncator.rb', line 12

def initialize(options={})
  @options = options
  @retentions = options[:retentions].keys
  @redis = Batsd::Redis.new(options )
  @diskstore = Batsd::Diskstore.new(options[:root])
end

Instance Method Details

- (Object) run(retention)

Perform a truncation run. Sole argument is the aggregation level to be truncated (e.g., 10, 60, 600)

If the retention interval is the first one, it's assumed to stored in redis, and the truncation is performed by zremrangebyscore on the zsets.

For other retention intervals, it's assumed that they are stored on disk. Each key is truncated on disk.

In neither case are gauges included for truncation.



30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'lib/batsd/truncator.rb', line 30

def run(retention)
  min_ts = Time.now.to_i - (@options[:retentions][retention] * retention)
  keys = @redis.datapoints(with_gauges=false)
  keys = keys.collect do |k|
    if (k.match(/^timer/) rescue false)
      ["mean", "min", "max", "upper_90", "stddev", "count"].collect{|a| "#{k}:#{a}"}
    else
      k
    end
  end.flatten

  if retention == @retentions.first
    # First retention is stored in redis, so just need to truncate the zset
    # TODO: can we do this in bulk with lua script?
    keys.each { |key| @redis.truncate_zset(key, min_ts) }
  else
    # Stored on disk
    keys.each do |key|
      key = "#{key}:#{retention}"
      @diskstore.truncate(@diskstore.build_filename(key), min_ts)
    end
  end
end