Oh, you mixed it with laminating resin? You could try spraying this resin instead if its not too fluid. The only difference between gelcoat and resin is a thixotrope like silica and fillers to make it hard or sandable.

It does sound like you are using a brush only gel.  Especially as youre upto 5.5mm nozzle, have thinned it and it still won't spray. 

Warren Penalver
Easy Composites / Carbon Mods - Technical Support Assistant