Irregularity
Irregularity is an umbrella term used by Kyubey for all beings whose existence he can't explain.
Until they are recognized, time travelers are usually classified as Irregulars. Such was the case for Homura Akemi, who was considered an Irregular due to the knowledge and powers she possessed, as well as Kyubey not recognizing her. Mabayu Aki was also considered an Irregular due to having wiped herself from Kyubey's memories.
Alternatively, a Magical Girl with unusually high potential, power and extra abilities due to another girl's wish is also considered an Irregular due to the unprecedented amount of power they possess. Madoka's own high potential categorized her as an Irregular and a similar case was Tart's second transformation gained during the fight against Corbeau. Another such case was Oriko achieving a new form after realizing her love for Kirika, which coincided with her wish.
During her final battle, Tart became something called the Perfect Irregular with the help of Riz's wish. This third, final form also allowed her to not become a Witch before her death despite her Soul Gem becoming something akin to a Grief Seed.
See also: Category:Irregulars for a list of known Irregulars.