What is the most difficult rudiment for you guys to pull off? Meaning, What is the hardest to get up to tempo with good sound quality and what not?
I would vote for triplet stroke roll; I have a hard time fitting the two hands together in time.

The hardest rudiment for me to get REALLY fast was the Swiss Army Triplet. The triplet stroke roll is my favorite type of roll. Those are so cool to throw in for the fun of it. I'm right now working on my 5 stroke roll to be as fast as my double stroke. I pretty much got my triple and 4 stroke rolls down as fast as my double. But triplet stroke rolls rock.

flammed mills and pataflaflas. If the devil was a rudiment, he would be the flammed mill. lol

I'm really trying to get this warmup right now from SCV called S&G which has rolls with accents. It's interesting since I have to slow it down a lot. Not usual for me.

The hardest rudiments for me to get up to speed and clean are inverted cheese. It feels kinda akward playing the accented diddle right after a tap on the same hand. For me anyways.

I'm not sure what this is called, maybe some of you experts could help me out with this, but basically it a flam accent with all parts diddled. Like a flam five but with the last partial diddled as well, they're pretty crazy, but i actually found a piece that has them written in on Devin Namaky's site. its called "street fame". they're towards the end.

any shifting of the accent during flam accents. keep the flam on the first hit but shift the accent, its nuts!

Of course there are riduculous rudiments that no line even tries like invereted cheesed backsticked malf taps or something stupid, but in terms of the real world and a line situation I would say ratamacues which you don't hear lines play much these days. It just seems there are so many places in this rudiment where people use different interpretations for the diddle in terms of spacing, or the three singled 24th notes, that to clean it and play it the same in a line over and over is very difficult.

inverted cheese is really hard for me to get up to speed and clean.

Flam para-diddle-diddles...or as we like to call them....mother fu**ers...lol Cuz they are a mother fu**er to play...