Same motor, same gearing, of course speed @6.6v > @4.8v > @3.7v !
I am currently using 2xA123 + 1/12 + 10.5R motor. Speed is very fast. No need to add booster. The speed is about same as 4NiMh + 5.5R.
2xA123 weights about 150g, so it is still very light compared to 4NiMh.
1 huge advantage of A123/LiFe is that you can charge them at up to 4C. (ie can use 10A to charge 2500mAh). It only takes about 15-20 mins for a full charge. So 1 set of LiFe is all you need @ the track.
One more advantage of A123/LiFe is that you can charge to 100% and store it, it will not become 'fat boy' battery, unlike LiPo. You can also store it for months without any ill-effects! So just charge them up when you have time.