Stupid coding mistakes
Made this mistake today while writing some code to delete data out of Redis.
def delete_keys
cursor = "0"
loop do
cursor, keys = redis.scan(cursor, count: SCAN_COUNT)
keys.reject! { |key| keep_key?(key) }
redis.del(keys) unless keys.empty?
break if cursor == "0"
end
end
def keep_key?(key)
keep_keys.any? { |keep_key| key.include?(keep_key) }
end
def keep_keys
flattened_keys << transaction_id
end
The mistake was in the final method keep_keys. Every check to see if a key should be rejected I was adding an element to the flattened_keys array over and over again, causing my deletion to slow down over time. A simple change to memoize the keep keys made the process go from never finishing to, completing in a few seconds.
def keep_keys
@keep_keys ||= (flattened_keys << transaction_id)
end