There are many metrics we can use to talk about the performance of a quantum device.  From the perspective of someone who would like to use one, good metrics should be predictive of the performance of the device in interesting applications (short depth circuit, error correction, etc).  In the context of IBM's superconducting quantum devices I will discuss a number of metrics and how they predict (or fail to predict) the real world performance.