Class: Batsd::Diskstore

Inherits:
Object
  • Object
show all
Defined in:
lib/batsd/diskstore.rb

Overview

Handles disk operations writing, truncating, and reading

Instance Method Summary (collapse)

Constructor Details

- (Diskstore) initialize(root)

Create a new diskstore object



9
10
11
 
# File 'lib/batsd/diskstore.rb', line 9

def initialize(root)
  @root = root
end

Instance Method Details

- (Object) append_value_to_file(filename, value, attempts = 0)

Append a value to a file

Open the file in append mode (creating directories needed along the way), write the value and a newline, and close the file again.



31
32
33
34
35
36
37
38
39
40
41
42
43
 
# File 'lib/batsd/diskstore.rb', line 31

def append_value_to_file(filename, value, attempts=0)
  FileUtils.mkdir_p filename.split("/")[0..-2].join("/")
  File.open(filename, 'a+') do |file|
    file.write("#{value}\n")
    file.close
  end
rescue Exception => e
  puts "Encountered an error trying to store to #{filename}: #{e} #{e.message} #{e.backtrace if ENV["VERBOSE"]}"
  if attempts < 2
    puts "Retrying #{filename} for the #{attempts+1} time"
    append_value_to_file(filename, value, attempts+1)
  end
end

- (Object) build_filename(statistic)

Calculate the filename that will be used to store the metric to disk.

Filenames are MD5 hashes of the statistic name, including any aggregation-based suffix, and are stored in two levels of nested directories (e.g., /00/01/0001s0d03dd0s030d03d)



20
21
22
23
24
 
# File 'lib/batsd/diskstore.rb', line 20

def build_filename(statistic)
  return unless statistic
  file_hash = Digest::MD5.hexdigest(statistic)
  File.join(@root, file_hash[0,2], file_hash[2,2], file_hash)
end

- (Object) read(statistic, start_ts, end_ts)

Reads the set of values in the range desired from file

Reads until it reaches end_ts or the end fo the file. Returns an array of {timestamp: ts, value: v} hashes.



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/batsd/diskstore.rb', line 50

def read(statistic, start_ts, end_ts)
  datapoints = []
  filename = build_filename(statistic)
  begin
    File.open(filename, 'r') do |file| 
      while (line = file.gets)
        ts, value = line.split
        if ts >= start_ts && ts <= end_ts
          datapoints << {timestamp: ts.to_i, value: value}
        end
      end
      file.close
    end
  rescue Errno::ENOENT => e
    puts "Encountered an error trying to read #{filename}: #{e}" if ENV["VVERBOSE"] 
  rescue Exception => e
    puts "Encountered an error trying to read #{filename}: #{e}"
  end
  datapoints
end

- (Object) truncate(filename, since)

Truncates a file by rewriting to a temp file everything after the since timestamp that is provided. The temp file is then renaemed to the original.



75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
 
# File 'lib/batsd/diskstore.rb', line 75

def truncate(filename, since)
  puts "Truncating #{filename} since #{since}" if ENV["VVERBOSE"]
  unless File.exists? "#{filename}tmp"  
    File.open("#{filename}tmp", "w") do |tmpfile|
      File.open(filename, 'r') do |file|
        while (line = file.gets)
          if(line.split[0] >= since rescue true)
            tmpfile.write(line)
          end
        end
        file.close
      end
      tmpfile.close
    end
    FileUtils.cp("#{filename}tmp", filename) rescue nil
  end
rescue Errno::ENOENT
  puts "Encountered an error trying to truncate #{filename}: #{e}" if ENV["VVERBOSE"] 
rescue Exception => e
  puts "Encountered an error trying to truncate #{filename}: #{e}"
ensure 
  FileUtils.rm("#{filename}tmp") rescue nil
end