Sometimes while I'm charging up for a spell, before I even use it, they disappear, I'll be using 0 Pip spells and when I look back, NO PIPS, Its possible i might've not noticed i used them, but i know a few times i havent even used a elemental blade yet, and they disappear.
I imagine they were stolen from you and you did not notice. At least that has what has happened to me a few times. I'll ask my partner. "Where did all my Pips go?" He usually says " Yea they stole them from you" LoL