Good to see you posting again, Nick.
I have no experience with the mask you're talking about, but I'd say if it doesn't look right to you, don't use it. I like to enlarge the schemes that come in the instructions (provided that they are correct) and cut them up into their different color sections. I start with one color, usually black because I'm lazy and can shoot it from a rattle can. Then I mask off the black areas and spray the second color. I like to keep the mask elevated very slightly from the model with putty or even little balls of tape. After that I mask again and spray the third color.
I know what you mean about those sand colors edges on the cammo. The first time I saw it I thought it was a new four color scheme. I think the best way to replicate it is with a very fine spray of sand. I'd like to see your result. I've always wanted to try it but have never gotten around to it.
Here's the result on a Paladin:
