Bias in algorithms is a costly human oversight--costly due to its immense impact on marginalized and underrepresented communities. As artificial intelligence (AI) continues to scale across industries and functions, it can be riddled with unconscious biases that do more harm than good. While AI ethicists and responsible AI practitioners often speak about the need for more transparency and accountability in the AI lifecycle, it is time for fire departments to consider the impact, and the benefit, of AI on their departments.