psx_gpu: fix line mask test