Class: Batsd::Redis

Inherits:
Object
  • Object
show all
Defined in:
lib/batsd/redis.rb

Overview

This is a thin wrapper around the redis client to handle multistep procedures that could be executed using Redis scripting

Instance Method Summary (collapse)

Constructor Details

- (Redis) initialize(options)

Opens a new connection to the redis instance specified in the configuration or localhost:6379



12
13
14
15
16
# File 'lib/batsd/redis.rb', line 12

def initialize(options)
  @redis = ::Redis.new(options[:redis] || {host: "127.0.0.1", port: 6379} )
  @redis.ping
  @retentions = options[:retentions].keys
end

Instance Method Details

- (Object) add_datapoint(key)

Stores a reference to the datapoint in the 'datapoints' set



125
126
127
# File 'lib/batsd/redis.rb', line 125

def add_datapoint(key)
  @redis.sadd "datapoints", key
end

- (Object) client

Expose the redis client directly



19
20
21
# File 'lib/batsd/redis.rb', line 19

def client
  @redis
end

- (Object) datapoints(with_gauges = true)

Convenience accessor to members of datapoints set



114
115
116
117
118
119
120
# File 'lib/batsd/redis.rb', line 114

def datapoints(with_gauges=true)
  datapoints = @redis.smembers "datapoints"
  unless with_gauges
    datapoints.reject!{|d| (d.match(/^gauge/) rescue false) }
  end
  datapoints
end

- (Object) extract_values_from_string(key)

Create an array out of a string of values delimited by <X>



79
80
81
82
# File 'lib/batsd/redis.rb', line 79

def extract_values_from_string(key)
  cmd = "local t={} ; local i=1\nlocal str = redis.call('get', KEYS[1])\nif (str) then\nfor s in string.gmatch(str, \"([^\"..\"<X>\"..\"]+)\") do\nt[i] = s\ni = i + 1\nend\nredis.call('del', KEYS[1])\nend\nreturn t\n"
  @redis.eval(cmd, 1, key.to_sym)
end

- (Object) get_and_clear_key(key)

Returns the value of a key and then deletes it.



68
69
70
71
# File 'lib/batsd/redis.rb', line 68

def get_and_clear_key(key)
  cmd = "local str = redis.call('get', KEYS[1])\nredis.call('del', KEYS[1])\nreturn str\n"
  @redis.eval(cmd, 1, key.to_sym)
end

- (Object) store_and_update_all_counters(timestamp, key, value)

Store a counter measurement for each of the specified retentions

  • For shortest retention (where timestep == flush interval), add the value and timestamp to the appropriate zset

  • For longer retention intervals, increment the appropriate counter by the value specified.

TODO: This can be done in a single network request by rewriting it as a redis script in Lua



34
35
36
37
38
39
40
41
42
43
# File 'lib/batsd/redis.rb', line 34

def store_and_update_all_counters(timestamp, key, value)
  @retentions.each_with_index do |t, index|
    if index.zero?
      @redis.zadd key, timestamp, "#{timestamp}<X>#{value}"
    else index.zero?
      @redis.incrby "#{key}:#{t}", value
      @redis.expire "#{key}:#{t}", t.to_i * 2
    end
  end
end

- (Object) store_raw_timers_for_aggregations(key, values)

Store unaggregated, raw timer values in bucketed keys so that they can actually be aggregated "raw"

The set of tiemrs are stored as a single string key delimited by x0. In benchmarks, this is more efficient in memory by 2-3x, and less efficient in time by ~10%

TODO: can this be done more efficiently with redis scripting?



59
60
61
62
63
64
65
# File 'lib/batsd/redis.rb', line 59

def store_raw_timers_for_aggregations(key, values)
  @retentions.each_with_index do |t, index|
    next if index.zero?
    @redis.append "#{key}:#{t}", "<X>#{values.join("<X>")}"
    @redis.expire "#{key}:#{t}", t.to_i * 2
  end
end

- (Object) store_timer(timestamp, key, value)

Store a timer to a zset



47
48
49
# File 'lib/batsd/redis.rb', line 47

def store_timer(timestamp, key, value)
  @redis.zadd key, timestamp, "#{timestamp}<X>#{value}"
end

- (Object) truncate_zset(key, since)

Truncate a zset since a treshold time



98
99
100
# File 'lib/batsd/redis.rb', line 98

def truncate_zset(key, since)
  @redis.zremrangebyscore key, 0, since
end

- (Object) values_from_zset(metric, begin_ts, end_ts)

Return properly formatted values from the zset



103
104
105
106
107
108
109
110
# File 'lib/batsd/redis.rb', line 103

def values_from_zset(metric, begin_ts, end_ts)
  begin
    values = @redis.zrangebyscore(metric, begin_ts, end_ts)
    values.collect{|val| ts, val = val.split("<X>"); {timestamp: ts, value: val } }
  rescue
    []
  end
end