On-chip L1 and L2 caches represent a sizeable fraction of the totalpower consumption of microprocessors. In deep sub-micron technology, the subthreshold leakage power is becoming the dominantfraction of the total power consumption of those caches. In thispaper, we present optimization techniques to reduce the leakagepower of on-chip caches assuming that there are multiple thresholdvoltages, VTH's, available. First, we show a cache leakage optimizationtechnique that examines the trade-off between access timeand leakage power by assigning distinct VTH's to each of the fourmain cache components - address bus drivers, data bus drivers, decoders, and SRAM cell arrays with sense-amps. Second, we showoptimization techniques to reduce the leakage power of L1 and L2on-chip caches without affecting the average memory access time. The key results are: 1) 2 VTH's are enough to minimize leakage in asingle cache; 2) if L1 size is fixed, increasing the L2 size can resultin much lower leakage without reducing average memory accesstime; 3) if L2 size is fixed, reducing L1 size can result in lower leakagewithout loss of the average memory access time; and 4) smallerL1 and larger L2 caches than are typical in today's processorsresult in significant leakage and dynamic power reduction withoutaffecting the average memory access time.