The ideal would be to work directly with libpng or libjpeg. It's true that they are libraries that, at first glance, seem complex and esoteric but they give you what it's all about - open an image of their format and give you a pointer to an array with the image data. An example file reading with libpng: link
If you do not want to work with the low level of the various image load libraries, you can use something that involves them in an abstraction layer, such as ITK, OpenCV, or VTK. image display). SDL would also be a good alternative and is simpler to use than the ones I mentioned above.
As for ease of installation, it depends on whether or not you use CMake. If you know how to use Cmake every c / c ++ library becomes easy to use, simply by the .h directory and the pro .lib path in Cmake according to his rules. If you do not know how to use Cmake, you will have problems.