While you put forth some argument for time and light, it falls short in a number of areas when it comes to panoramic photography.
Time is relevant to not only shooting but also time needed in stitching and correct control point placement. The fact you also must tilt the panohead every time you need to shoot the Zenith and then Nadir shots.
As for the f/2.8 vs f/3.5 we are only talking one f stop in total. Sure the extra light is useful in "normal" photography but when it come to shooting professional panoramas you will never use the lens wide open as this effects the total depth of field. Panoramas require focus from near to infinity in just about all occasions. The sweet spot for the Sigma 10mm Fisheye is f/11.0 and Sigma 8mm is f/8.0 these are well known and adopted facts. Potentially f/5.6 at a stretch, this remains a long way from the f/2.8.
On another note the Sigma 10mm suffers badly from Chromatic Aberration and most in the know avoid this lens if favour of other options. If you heart is set on a lens that has more universal uses outside and along with panoramic photograph I would suggest you look at the Tokina 10-17mm Zoom Fisheye lens. Which you will find of higher quality and priced extremely competitively.
If my word is not enough I suggest you visit many other professional panorama photography forums and seek further reassurance on the matter.
The one thing I can tell you to avoid is advise from camera store sales people on fisheye lens advise. Their knowledge is limited in this very specialised field.