In theory it would work but I wouldn't want to, pastels use a binder called gum tragacanth to bind them and it may affect the properties of the paint, I have used chalk pastels and white spirit as a substitute to pigments though!