Ahlulbayt, is a term used for"wives" but why is it sometimes exclusively used for a specific group?

Saad bin Abi Waqas relates that when the Ayat-e-Mubahila was sent down, Prophet Mohammad(sawaw) called Imam Ali(a.s.), Bibi Fatima(s.a.), Imam Hasan(a.s.) and Imam Hussain(a.s.) and said "O my Allah(swt), these are my Ahl-e-Bait" . (Sahih Muslim, Vol.2, Page 287)