Hi, Thanks for commenting. I also had a similar problem with my D800 and found that menu setting for jpg was Basic, plus copies of View and Capture NX2 were long in tooth and seemed buggy since Nikon long ago discontinued support. I downloaded fresh copies and reset Image to RAW + Fine and got my setup back on track to generating images that pass the QA size criteria. I like the colors with NX2 and the control points.
Shooting Raw 14 bits with Nikon D5300 compressed, I get images that are 1080 x 1620 when I save as jpg with Nikon Capture NX2 which are too small to be submitted to Alamy stock photography: What am I missing as Nikon specs rate D5300 image size up to 6000 x 4000 L?